21 of 21 Data Pipeline Jobs in Lancashire

Job summary The role holder is responsible for the leading the design, development, and ongoing enhancement of organisations data infrastructure and pipelines to support advanced data collection, storage, processing, and analysis. Main duties of the job The role holder is accountable for a team of data engineers, fostering a culture of technical excellence and … continuous improvement. The role holder will work collaboratively with cross-functional teams, including analysts, software engineers, and key stakeholders, to ensure that data solutions are robust, scalable, and aligned with the strategic goals of delivering high-quality care for our patients. About us Since 2012, CHEC has been working with the NHS to increase patient choice and provide … engineering solutions and provide comprehensive reports. Identify and address performance bottlenecks, optimise data processing and query performance for scalability. Implement monitoring frameworks to oversee data pipeline health, swiftly resolving issues to maintain system integrity. Conduct performance tuning to fulfil scalability and availability targets. Develop and enforce stringent data security measures, including access controls More ❯

Lead data engineers at Thoughtworks develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. They might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to … a program inception to build a new product. Alongside hands-on coding, they are leading the team to implement the solution. Job responsibilities You will lead and manage data engineering projects from inception to completion, including goal-setting, scope definition and ensuring on-time delivery with cross team collaboration. You will collaborate with stakeholders to understand their strategic … objectives and identify opportunities to leverage data and data quality. You will design, develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. You will be responsible to create, design and develop intricate data processing pipelines, addressing clients' most challenging problems. More ❯

Data Engineer - Leading Pharmaceutical Company - Manchester(Tech Stack: Data Engineer, Databricks, Python, Power BI, Azure, TSQL, ETL, Agile Methodologies)About the Role: We are seeking a talented and experienced Data Engineer on behalf of our client, a leading Software House. This is a fully remote position, offering the opportunity to work with cutting-edge … technologies and contribute to exciting projects in a collaborative environment.About the Role: Our client is seeking an experienced Data Engineer to join their team in Manchester. This hybrid position involves working within the pharmaceutical industry, focusing on the design, development, and maintenance of data pipelines, ETL processes, and databases. The role is ideal for someone passionate … about improving processes, ensuring data quality, and maintaining compliance with regulatory standards. focusing on designing, developing, and maintaining data pipelines, ETL processes, and databases. If you are passionate about driving continuous improvement and ensuring data quality and compliance, we want to hear from you.Key Responsibilities:Design, develop, maintain, and optimise data pipelines More ❯

Snowflake Developer Location: ManchesterDuration: 3 Days a WeekJob Type: Permanent/FTE As a Snowflake Developer, you'll be a key player in designing, developing, and maintaining our data solutions. You'll work with cutting-edge technologies, leveraging your expertise in Snowflake to build robust and scalable data pipelines and architectures. This role is perfect for … SQL: Possess a deep understanding of Snowflake's architecture and internals, including roles, dynamic tables, streams, and tasks. You'll need excellent skills in writing complex SQL queries. Data Modeling: Extensive working knowledge of various data models, including Dimensional Data Model , ER Data Model , and Data Vault . Cloud Computing … Strong foundational knowledge of cloud computing principles. Data Expertise: Significant experience in data-related projects and applications, including a good understanding of data management and data governance . Programming & Tools: Good working knowledge of Python and GitLab . Stakeholder Management: Proven experience in managing product owners and stakeholders. Soft Skills: A great More ❯

About the Role Reporting to the Head of Data, Insights & Analytics you will lead a team of Data Engineers in the design, development and maintenance of Travel Counsellors’ data infrastructure. This role will be a high impact, multi-faceted working on data pipelines, our new strategic Data Platform and operationalising … our analytical/data science solutions. Principal Accountabilities Support the design and implementation of data pipelines to ensure accurate, consistent and timely data across our analytical and reporting needs Work closely with Data Scientists, Analysts and business SMEs to automate and scale our machine learning and AI capabilities Support the development of … our new cloud-based Data Platform, transforming systematic data into a business view for reporting and analytics Take ownership of our existing Microsoft technology (SQL Server, SSIS, SSAS, SSRS) whilst delivering on our migration plans to cloud-based technology Create and maintain comprehensive documentation for data processes and architectures. Provide training and support to More ❯

every decision wemake and action we take, guiding us to deliver impact how and where it mattersmost . Connect to your opportunity We're looking for a skilled Data Lake Reporting Assistant Manager to join our team and play a key role in developing and maintaining our data lake reporting environment. You'll work closely with … experienced data analysts and engineers to deliver impactful reports that help us understand our business better. Key responsibilities include: Collaborate closely with senior data analysts and engineers, contributing to the development and implementation of our reporting strategy. Learn and apply industry best practices for data warehousing, reporting, and visualization. Data Lake Reporting … Environment: Become proficient in navigating and understanding data lake architecture. Assist in implementing data quality checks and validation processes to ensure accurate and reliable reporting. Work alongside data engineers to understand and optimize data pipelines for reporting needs. Translate business requirements into technical specifications for reports and dashboards, working with senior team More ❯

with a leading UK consumer goods brand, known nationwide for their products in the home & lifestyle space. With ongoing growth and digital transformation, they are investing heavily in data capabilities to improve business decision-making, customer insight, and operational efficiency. The organisation has already built out a small but capable data team, and they're now … looking to appoint a BI Lead to accelerate their reporting, analytics, and automation strategy.This is a high-impact role where you'll shape how data is delivered across the group, working with stakeholders across Finance, Operations, Marketing, and Commercial to enable smarter, faster decisions. The Role This newly created BI Lead role sits within a growing data … at Harnham via the Apply link on this page. Keywords BI Lead, Business Intelligence Lead, BI Developer, BI Specialist, Data Engineer, Power BI, Azure, Databricks, SQL, Pipelines, Data Visualisation, Self-Service Analytics, Reporting Platform, Consumer Goods, Lancashire, Preston, Burnley, North West, UK Jobs More ❯

with a leading UK consumer goods brand, known nationwide for their products in the home & lifestyle space. With ongoing growth and digital transformation, they are investing heavily in data capabilities to improve business decision-making, customer insight, and operational efficiency. The organisation has already built out a small but capable data team, and they’re now … looking to appoint a BI Lead to accelerate their reporting, analytics, and automation strategy. This is a high-impact role where you’ll shape how data is delivered across the group, working with stakeholders across Finance, Operations, Marketing, and Commercial to enable smarter, faster decisions. The Role This newly created BI Lead role sits within a growing data … at Harnham via the Apply link on this page. Keywords BI Lead, Business Intelligence Lead, BI Developer, BI Specialist, Data Engineer, Power BI, Azure, Databricks, SQL, Pipelines, Data Visualisation, Self-Service Analytics, Reporting Platform, Consumer Goods, Lancashire, Preston, Burnley, North West, UK Jobs More ❯

with a leading UK consumer goods brand, known nationwide for their products in the home & lifestyle space. With ongoing growth and digital transformation, they are investing heavily in data capabilities to improve business decision-making, customer insight, and operational efficiency. The organisation has already built out a small but capable data team, and they’re now … looking to appoint a BI Lead to accelerate their reporting, analytics, and automation strategy. This is a high-impact role where you’ll shape how data is delivered across the group, working with stakeholders across Finance, Operations, Marketing, and Commercial to enable smarter, faster decisions. The Role This newly created BI Lead role sits within a growing data … at Harnham via the Apply link on this page. Keywords BI Lead, Business Intelligence Lead, BI Developer, BI Specialist, Data Engineer, Power BI, Azure, Databricks, SQL, Pipelines, Data Visualisation, Self-Service Analytics, Reporting Platform, Consumer Goods, Lancashire, Preston, Burnley, North West, UK Jobs More ❯

We are seeking a Senior Backend Python Developer to support the development of our AI-driven marketing platform, with a strong focus on performance, scalability, and data pipeline architecture. This role is ideal for someone experienced in backend API development and working with modern data infrastructure. At Ask BOSCO , we're on a mission to … Backend Python Developer, you won’t just write code - you’ll shape the future of a high-growth MarTech product, lead technical strategy, and work directly with founders, data scientists, and clients to deliver impactful, intelligent solutions. If you're excited by ownership, innovation, and the chance to make a real mark in a scaling business, this is … of profits donated to 1moreChild orphanage yearly Markets: Marketing & eCommerce Agencies. eCommerce retailers. Your Expertise: Highly experienced Developer with experience of backend API development and working with modern data infrastructure. At ASK BOSCO , part of Modo25, we’re building a smarter, faster, people-first marketing tech business. We're entering a phase of accelerated growth, and you’ll More ❯

are to drive a modern digital government, by: joining up public sector services harnessing the power of AI for the public good strengthening and extending our digital and data public infrastructure elevating leadership and investing in talent funding for outcomes and procuring for growth and innovation committing to transparency and driving accountability We are home to the Incubator … for Artificial Intelligence (I.AI), the world-leading GOV.UK and at the forefront of coordinating the UK's geospatial strategy and activity. We lead the Government Digital and Data function and champion the work of digital teams across government. We're part of the Department for Science, Innovation and Technology (DSIT) and employ more than 1,000 people all … difference across the UK. You'll solve some of the nation's highest-priority digital challenges, helping millions of people access services they need. The GDS IDEA (Insight, Data Science, Economics, and Analysis) Unit is looking for talented data scientists to join our dynamic, high-impact, multidisciplinary team. We combine cutting-edge AI, advanced data More ❯

teams. Cloud Services: Experience with cloud platforms like AWS, GCP, or Azure. DevOps Tools: Familiarity with containerization (Docker) and infrastructure automation tools like Terraform or Ansible. Real-time Data Streaming: Experience with Apache Pulsar or similar systems for real-time messaging and stream processing is a plus. Data Engineering: Experience with Apache Spark, Databricks, or similar … big data platforms for processing large datasets, building data pipelines, and machine learning workflows. Workflow Orchestration: Familiarity with tools like Apache Airflow or Temporal for managing workflows and scheduling jobs in distributed systems. Stream Processing: Experience with Apache Flink or other stream processing frameworks is a plus. Desired Skills Asynchronous Programming: Familiarity with asynchronous programming tools More ❯

smooth and efficient functioning and development of our databases and pipelines. You will be responsible for developing, optimising and managing these databases and ensuring a high-level of data quality and integrity. You will be defining the principles and processes that underpin data management while ensuring we maintain the highest standards for data management … repeatable and scalable. Create and manage database schemas, tables, stored procedures, indexes, and triggers. Ensuring high availability of databases through clustering, replication, and other techniques. Implement and maintain data pipelines and ETL processes for real-time and batch data ingestion. Monitor database performance and proactively address issues related to tuning and optimization. Ensure data security, privacy, integrity, and compliance with internal and external regulations. Collaborate with third-party suppliers, software developers, data analysts, DevOps, and other stakeholders to meet project requirements. Plan and implement database backups, restores, and disaster recovery strategies. Taking an active role in supporting and in finding the root cause of operational issues and preventing recurrences. Support More ❯

Role: Azure Senior Data Engineer Location: Manchester, Lancashire Hybrid working: go into office 1x day a week Salary: £60,000 - £70,000 (dependent on experience) Insight into the Company: A large retail organisation - market leaders in their industry - are looking for a Senior Data Engineer to enter their team. You will be working in a small … team of 3 (including you!) and they are looking to build up the team over the next year. The ideal candidate will have experience with managing data warehouses from start to finish and liaise with IT teams regularly. The role will involve stakeholder management, working with diverse roles, such as software engineers, sales and marketing professionals and more. … Therefore, communicating your projects needs and understanding others is an essential part of this role. Role and Responsibilities: You will design, build and upgrade data pipelines You will work in CI/CD and with Software/DevOps teams in the organisation You will have expertise in azure - from collecting, to transforming to loading! Skills and Experience: Essential More ❯

Overview An excellent opportunity for a talented and motivated Product Manager to guide and support the delivery of the set of Science Data Processing (SDP) pipelines for the Mid and Low SKA Telescopes. The role is internally known as SKA Operations Scientist/SAFe Product Manager. At the SKA Observatory, Product Managers … are at the heart of turning vision into reality. In this role, you will support engineering and development teams as they deliver the Science Data Processor (SDP) pipeline software for both the Mid and Low telescopes. Your job is to help teams ask the right questions and solve the right problems. The goal is simple: ensure that … product goals, impacts, and deliverables. Ensure that the pipelines integrate effectively with telescope systems, data archives, and international science data networks. Drive continuous improvement in pipeline reliability, usability, and scalability to meet the challenges of petabyte-scale data processing. Represent the product to internal and external stakeholders, providing clear communication of progress, risks More ❯

is unique. We'd love you to join us. Delivery. Work. Projects. As a Digital Performance Analyst in DWP Digital's Specialist Devices team, you'll help turn data into decisions that improve the tools and services our colleagues rely on every day. You'll be part of a user-focused team that owns the end-to-end … ll collaborate with product owners, engineers, and designers to continuously improve services based on evidence and user needs. You'll also support wider government priorities around digital inclusion, data transparency, and service efficiency. It's a great opportunity to work at the heart of digital transformation in the UK's largest government department, where your analysis will help … to build insightful dashboards and reports that drive decision-making. SQL Proficiency - You're confident writing custom SQL to perform calculations, join multiple datasets, and feed clean, structured data into dashboards and analysis tools. Digital Workplace Insight - You've worked in digital workplace environments and know how to analyse data from digital systems to surface trends More ❯

web applications, focusing on lead generation workflows. Optimize and scale platform performance to handle increasing lead volumes efficiently. Integrate and manage APIs for customer eligibility checks, ensuring encrypted data transmission and receipt in compliance with GDPR and FCA guidelines. Collaborate with marketing teams and partners to enhance lead qualification processes using automated systems. Build responsive, conversion-driven user … interfaces with a seamless customer journey. Ensure platform security, including data encryption and compliance with UK regulatory standards. Troubleshoot, debug, and upgrade existing systems to improve functionality and user experience. Work with cloud infrastructure (AWS or Replit) to deploy, manage, and scale applications. Participate in code reviews, testing, and documentation to maintain high-quality standards. Stay updated on … and Express.js for server-side logic. Familiarity with Formik and Yup for form handling and validation. Database and Storage : Experience with Supabase or similar PostgreSQL-based solutions for data management. Ability to integrate and manage relational databases securely. API Integration : Strong experience in working with RESTful APIs (e.g., Numverify, ZeroBounce) to send and receive encrypted data. Knowledge of More ❯

plus 28.9% employer pension contributions, hybrid working with 60% home working, flexible hours, and great work life balance. Would you like to kickstart a career in data and technology while gaining a recognised qualification in a supportive, diverse environment? DWP. Digital with Purpose. We're offering a fantastic opportunity to join our Level 5 Data Engineering … UK's largest and most impactful digital organisations. At DWP Digital, we design and build the technology behind vital services like Universal Credit, pensions, and child maintenance. Our data engineers play a key role in making sure these services are powered by reliable, high-quality data, helping teams make better decisions and improving outcomes for millions … of people. As an apprentice, you'll join our Data Platforms and Operations team , working with cloud technologies like AWS and tools such as Informatica. You'll learn how to build and maintain data pipelines, support platform migrations, and help deliver data products that are secure, efficient, and user-focused. You'll be supported More ❯

Testing Team with high quality QA Testers in preparation for project work commencing in October. About the Programme You ll join a Transition & Transformation programme migrating systems and data across a financial services landscape (mortgages, savings, core banking and digital/mobile). You ll work day-to-day with our delivery team and our clients SMEs to … plan, execute and assure end-to-end testing across applications, integrations and data pipelines. What You ll Do Execute E2E testing: Author and run functional, system, SIT, UAT support and regression suites across web, mobile (iOS/Android), and back-office systems. Defect management: Triage, track and report defects (Jira/Azure DevOps), drive resolution with development and … SME teams. Data migration testing: Validate mappings, perform sample-based and full reconciliations, prove data quality and completeness before/after cutover Non-functional support: Contribute to performance, reliability and basic security test activities (e.g., JMeter, OWASP awareness). Environment & test data: Coordinate environments, seed/refresh masked datasets, maintain repeatable test data More ❯

Testing Team with high quality QA Testers in preparation for project work commencing in October. About the Programme You’ll join a Transition & Transformation programme migrating systems and data across a financial services landscape (mortgages, savings, core banking and digital/mobile). You’ll work day-to-day with our delivery team and our clients SMEs to … plan, execute and assure end-to-end testing across applications, integrations and data pipelines. What You’ll Do Execute E2E testing: Author and run functional, system, SIT, UAT support and regression suites across web, mobile (iOS/Android), and back-office systems. Defect management: Triage, track and report defects (Jira/Azure DevOps), drive resolution with development and … SME teams. Data migration testing: Validate mappings, perform sample-based and full reconciliations, prove data quality and completeness before/after cutover Non-functional support: Contribute to performance, reliability and basic security test activities (e.g., JMeter, OWASP awareness). Environment & test data: Coordinate environments, seed/refresh masked datasets, maintain repeatable test data More ❯

simply health cash plan, an annual discretionary bonus, plus a range of partnership and lifestyle discounts. Your Impact Lead the design, implementation and delivery of software applications and data … integrations aligned to enterprise architecture standards. Take full ownership for documenting systems, workflows and technical processes, ensuring your work is well understood, maintainable and transferable. Guide CI/CD pipeline development and DevOps practices, with a focus on quality, reliability and automation. Develop integration patterns using APIs, data pipelines or messaging systems. Collaborate with the Architecture team More ❯