1 to 25 of 82 Data Pipeline Jobs in the North West

Head of Data Engineering - Preston A Head of Data Engineering/Data Engineering Manager to lead the design, development, and enhancement of the data infrastructure and pipelines is required by a leading company based in Preston. The role offers Hybrid working, so 2 - 3 days in the office a week. You must … solutions for delivering high-quality care. The ideal candidate will lead the design and execution of cloud-based based scalable data storage solutions, oversee ETL pipeline development and optimisation, establish and manage data schemas and dictionaries, develop data integration solutions, and lead data cleansing, validation, and enrichment processes. Duties … You'll be on the lookout for performance bottlenecks and working to optimise data processing and query performance. This will help make sure your data pipeline is scalable and ready to handle big loads. You'll also be setting up monitoring frameworks to keep an eye on how your data pipeline is More ❯

we're proud to be consumer champions in the automotive space. We're constantly exploring smarter ways to connect people with the right cars - and that's where data engineering plays a key role. You'll join a focused data team of 4 professionals (2 Data Scientists, 1 BI Developer, and yourself as Data … CI/CD pipelines for our data infrastructure, ensuring automated testing and smooth deployments. Participating in code reviews and contributing to team standards for data pipeline development. Keeping on top of the latest datasets available within the automotive space and making recommendations about new data sources. Supporting and expanding our Microsoft Azure infrastructure … optimising it for data pipeline performance. Writing comprehensive data documentation for analytics-focused entities, accelerating your colleagues' understanding of available data. What You'll Need to Succeed Essential Requirements: 2-4 years of experience building robust data pipelines in a commercial environment or through complex personal projects. Strong Python skills including experience with More ❯

algorithms. Lead the selection, validation, and optimization of models to discover meaningful patterns and insights, ensuring models remain relevant, reliable, and scalable. Drive continuous integration and deployment of data science solutions, optimizing performance through advanced machine learning techniques, code reviews, and best practices. 'Develop and deliver sophisticated visualizations, dashboards, and reports translate complex data into clear … actionable insights for business stakeholders. Present technical solutions to business stakeholders, using creative methods to explain complex concepts, increase understanding, and encourage solution adoption. Mentor and develop junior data scientists, fostering a culture of continuous learning, knowledge sharing, and skills development within the organization. Write clean, high-quality code, ensuring all outputs pass quality assurance checks, and contribute … Experience with MLOps practices including model drift detection, decay, A/B testing, integration testing, differential testing, Python package building, and code version control. Skilled in data pipeline creation and working with both structured and unstructured data. Familiar with cloud platforms (AWS, Azure, GCP) and containerization technologies like Docker and Kubernetes. Excellent problem-solving skills, combined with More ❯

Remote Data Architect sought by a well-established company in their industry; our client is seeking an adept professional to spearhead strategic leadership and technical expertise within their data solutions landscape, focussed on cloud data platforms, but including IoT analytics, data integration, and migration projects. This unique opportunity is offered on a … remote-first basis, but with an open invitation to their headquarters any time. Role Responsibilities: The Data Architect will be pivotal in shaping the approach to designing and developing data solutions using cloud technologies such as AWS and Azure. This position involves: - Leading the design and implementation of innovative cloud data platforms. - Setting standards … using Azure, AWS, or exceptional on-premise design expertise. - At least 5 years in data engineering or business intelligence roles. - Proficiency in ETL and data pipeline design, with a technology-agnostic approach. - A solid understanding of data warehouse and data lake principles. - Expert SQL skills and demonstrable data modelling More ❯

Following the acquisition of Brookson by People2.0, the responsibilities of the Data and Analytics team have significantly expanded. We are now seeking a skilled Data Engineer to join our dynamic team. As a Data Engineer within the Data and Analytics team, you will play a pivotal role in designing, maintaining, and optimising … the data architecture for People2.0 Group. This architecture ensures the seamless flow of data from source systems to the Data Warehouse, providing a robust foundation of aggregated analytical base tables and operational sources. Our data architecture is critical in enabling People2.0 Group to become a data-driven organisation. Your responsibilities … will include maintaining existing data pipelines, developing new branches within the architecture, and collaborating closely with stakeholders across the business. You will engage with stakeholders from various regions, including EMEA, US, and APAC, ensuring that the architecture aligns with regional and global requirements. Reporting directly to the Senior Data Engineer within the Data and More ❯

Job Title: Data Product Manager - Fully Remote About the Company: Join a leading AI-driven company that helps enterprise brands and agencies optimise paid search and ads. Role Overview: We are looking for a technically strong, hands-on Data Product Manager to drive the evolution of our data platform and machine learning ecosystem. This … is a platform-focused role, deeply technical but not architectural, working across high-volume data pipelines and predictive ML models to improve data quality, scale, and performance. Key Responsibilities: Own the data product roadmap and define the vision for the data platform. Manage internal data projects with high customer impact. … Ensure integrity, quality, and performance of data models and pipelines. Collaborate closely with Data Engineering, Data Science and PMs Contribute to AI and GenAI initiatives in search intelligence. Key Requirements: 5-10 years of experience in data roles, with at least 3 years in Data Engineering or Data More ❯

Test Engineer Data Platforms (FTC-6months Hybrid | London or Manchester) My client is looking for an experienced Test Engineer to take ownership of testing across modern data platforms. This is a hands-on role where you’ll design, implement, and execute test strategies for data pipelines, models, and reporting solutions built on Azure technologies. … Key Responsibilities Define and own the overall testing strategy for data pipelines, data models, and BI solutions. Develop and implement automated testing frameworks integrated with CI/CD pipelines. Design and execute test cases across data ingestion, transformation, storage, and presentation layers. Validate data accuracy, performance, and usability within Azure Databricks, Azure … Data Factory, Azure Data Lake Storage, and Power BI. Establish and monitor data quality checks, reconciliation, and compliance processes. Collaborate closely with data engineers, BI developers, architects, and product owners to embed testing best practices across delivery. Essential Skills & Experience Proven experience as a Test Engineer within data engineering or More ❯

Data Engineer (Legal sector) Greater Manchester Hybrid (2 – 3 days per month in the office) £38,000 - £40,000 Only candidates within a commutable distance of Greater Manchester will be considered. About the Role Our legal client is seeking a Data Engineer to join their Data Team to design, build, and maintain an Azure … based data environment. Within this role you will architect and manage the data ecosystem, develop pipelines, oversee the data lake and enable high-performance reporting through Power BI. You’ll collaborate with Stakeholders to deliver the data roadmap, ensure a single source of truth, and help define a robust data strategy. This is a great opportunity to bring fresh ideas and shape the future of data within a growing organisation. Key Responsibilities Work with the Head of Department to prioritise and deliver data requirements into the data lake. Build, schedule, and manage data pipelines from operational systems into Azure. Maintain More ❯

Data Engineer Hybrid (2 – 3 days per month in the office) Greater Manchester £38,000 - £40,000 DOE Only candidates within a commutable distance of Greater Manchester will be considered. About the Role Our legal client is seeking a Data Engineer to join their Data Team to design, build, and maintain an Azure-based … data environment. Within this role you will architect and manage the data ecosystem, develop pipelines, oversee the data lake and enable high-performance reporting through Power BI. You’ll collaborate with Stakeholders to deliver the data roadmap, ensure a single source of truth, and help define a robust data strategy. … This is a great opportunity to bring fresh ideas and shape the future of data within a growing organisation. Key Responsibilities Work with the Head of Department to prioritise and deliver data requirements into the data lake. Build, schedule, and manage data pipelines from operational systems into Azure. Maintain both production and More ❯

Data Engineer Hybrid (2 – 3 days per month in the office) Greater Manchester £38,000 - £40,000 DOE Only candidates within a commutable distance of Greater Manchester will be considered. About the Role Our legal client is seeking a Data Engineer to join their Data Team to design, build, and maintain an Azure-based … data environment. Within this role you will architect and manage the data ecosystem, develop pipelines, oversee the data lake and enable high-performance reporting through Power BI. You’ll collaborate with Stakeholders to deliver the data roadmap, ensure a single source of truth, and help define a robust data strategy. … This is a great opportunity to bring fresh ideas and shape the future of data within a growing organisation. Key Responsibilities Work with the Head of Department to prioritise and deliver data requirements into the data lake. Build, schedule, and manage data pipelines from operational systems into Azure. Maintain both production and More ❯

DV Cleared Data Engineers – Manchester – AWS or Azure – Senior/Principal We’re supporting a leading global consultancy with the expansion of their secure data engineering team, and we’re looking for DV Cleared Data Engineers to join major projects in Manchester. If you hold active DV clearance and want to work on cutting … edge, mission-critical programmes in a highly collaborative technical environment, this is an excellent opportunity. They are considering Data Engineers at all levels from Junior up to Principal Data Engineer. What You’ll Be Doing Designing and delivering secure, high-performance data pipelines and analytics platforms Working across AWS or Azure environments, building scalable … national infrastructure and security programmes with real impact What We’re Looking For Active DV clearance (essential) Strong background in data engineering, ETL, and data pipeline development Experience with AWS or Azure cloud technologies Excellent coding skills in Python, SQL, or Scala Understanding of data warehousing, modelling, and streaming technologies Confident working in More ❯

Job Title: Airflow/AWS Data Engineer Location: Manchester Area (3 days per week in the office) Rate: Up to £400 per day inside IR35 Start Date: 03/11/2025 Contract Length: Until 31st December 2025 Job Type: Contract Company Introduction: An exciting opportunity has become available with one of our sector-leading financial services clients. … resource optimization for Airflow workers and AWS resources. Manage integrations with source control systems (GitHub, GitLab) and CI/CD pipelines. Collaborate with AWS teams and internal stakeholders for pipeline scalability and optimization. Design and implement process improvements, including automation, data delivery optimization, and infrastructure re-design. Develop ETL pipelines and data workflows using AWS … and tools like Jenkins, GitHub Actions, or GitLab CI. Solid understanding of containerization technologies (Docker, Kubernetes). Working knowledge of Python and SQL for automation and data pipeline development. Familiarity with monitoring and observability tools (Grafana, Prometheus, CloudWatch). Strong grasp of data architecture principles and ETL design patterns. Financial services or regulated industry experience More ❯

Data Engineer Location: Bolton (Hybrid – 2–3 days on-site per week) Salary: Up to £55,000 (depending on experience) Are you a skilled Data Engineer with a passion for Generative AI and emerging data technologies? This is your opportunity to play a key role in a growing international and cross-functional environment, supporting … the design, development, and optimisation of data pipelines that power next-generation AI systems across a world-class defence organisation. The Role As a Data Engineer (Generative AI) , you’ll be responsible for evaluating, building, and maintaining high-quality data sets for internal customers—ensuring performance, reliability, and maintainability at every stage. You’ll … work within MBDA’s IM GenAI Delivery Office , contributing to the design and deployment of resilient, secure, and responsive data pipelines supporting AI and NLP initiatives. You’ll collaborate closely with internal stakeholders, leveraging your expertise in structured and unstructured data to ensure compliance with MBDA’s data governance standards, while actively shaping the More ❯

Data Engineer Location: Bolton (Hybrid – 2–3 days on-site per week) Salary: Up to £55,000 (depending on experience) Are you a skilled Data Engineer with a passion for Generative AI and emerging data technologies? This is your opportunity to play a key role in a growing international and cross-functional environment, supporting … the design, development, and optimisation of data pipelines that power next-generation AI systems across a world-class defence organisation. The Role As a Data Engineer (Generative AI) , you’ll be responsible for evaluating, building, and maintaining high-quality data sets for internal customers—ensuring performance, reliability, and maintainability at every stage. You’ll … work within MBDA’s IM GenAI Delivery Office , contributing to the design and deployment of resilient, secure, and responsive data pipelines supporting AI and NLP initiatives. You’ll collaborate closely with internal stakeholders, leveraging your expertise in structured and unstructured data to ensure compliance with MBDA’s data governance standards, while actively shaping the More ❯

Data Delivery and Integration Manager - Full time permanent role - Retail industry - Remote/home-based - Salary up to £75,000 plus bonus, private healthcare and more! A large and growing retail business are looking to expand their IT function with the addition of a Data Integration Manager. This role will focus on end to end data engineering and integration solutions for a company wide modernisation programme. Duties will include: - Managing the processes for data engineering and data integration in order to align to business needs - Leading a team across engineering and integration - Work to ensure the data infrastructure of the business effectively supports analytics, AL, machine learning ML … and reporting - Designing of data pipelines, ETL, ELT processes and architecture - Work with internal stakeholders and multiple 3rd parties to seamlessly integrate systems, platforms and applications To be considered suitable for this Data Engineering Manager role you will need to have experience across the following: - Team leadership and guidance - Data engineering and aligning of More ❯

Data Architect Location: Manchester (Hybrid – Onsite Tuesdays) Salary: £85,000 - £90,000 + up to 25% Bonus A leading household name in the digital and e-commerce space, with over 16 million users on their app and £1 billion in annual revenue generated through digital channels, is embarking on a major 5-year data transformation strategy. … company-wide on-prem to Azure migration. What You’ll Be Doing Lead the cloud architecture transformation (Azure focus) Work hands-on with SQL, Azure, and data pipeline development Engage with stakeholders across the business – educate, influence, and lead Collaborate with BI, Data Engineering, and Digital Analytics teams Design and deliver scalable architecture for massive … data sets and high-volume user bases What You’ll Bring Minimum 2 years’ experience in architecture (cloud/data focused) Strong hands-on technical background (SQL, Azure essential) Experience in on-prem to cloud migration (Azure preferred) Ability to work autonomously, with strong stakeholder management Experience working with large data volumes and customer More ❯

Data Architect Location: Manchester (Hybrid - Onsite Tuesdays) Salary: £85,000 - £90,000 + up to 25% Bonus A leading household name in the digital and e-commerce space, with over 16 million users on their app and £1 billion in annual revenue generated through digital channels, is embarking on a major 5-year data transformation strategy. … company-wide on-prem to Azure migration. What You'll Be Doing Lead the cloud architecture transformation (Azure focus) Work hands-on with SQL, Azure, and data pipeline development Engage with stakeholders across the business - educate, influence, and lead Collaborate with BI, Data Engineering, and Digital Analytics teams Design and deliver scalable architecture for massive … data sets and high-volume user bases What You'll Bring Minimum 2 years' experience in architecture (cloud/data focused) Strong hands-on technical background (SQL, Azure essential) Experience in on-prem to cloud migration (Azure preferred) Ability to work autonomously, with strong stakeholder management Experience working with large data volumes and customer More ❯

Data Architect Location: Manchester (Hybrid – Onsite Tuesdays) Salary: £85,000 - £90,000 + up to 25% Bonus A leading household name in the digital and e-commerce space, with over 16 million users on their app and £1 billion in annual revenue generated through digital channels, is embarking on a major 5-year data transformation strategy. … company-wide on-prem to Azure migration. What You’ll Be Doing Lead the cloud architecture transformation (Azure focus) Work hands-on with SQL, Azure, and data pipeline development Engage with stakeholders across the business – educate, influence, and lead Collaborate with BI, Data Engineering, and Digital Analytics teams Design and deliver scalable architecture for massive … data sets and high-volume user bases What You’ll Bring Minimum 2 years’ experience in architecture (cloud/data focused) Strong hands-on technical background (SQL, Azure essential) Experience in on-prem to cloud migration (Azure preferred) Ability to work autonomously, with strong stakeholder management Experience working with large data volumes and customer More ❯

Data Architect Location: Manchester (Hybrid – Onsite Tuesdays) Salary: £85,000 - £90,000 + up to 25% Bonus A leading household name in the digital and e-commerce space, with over 16 million users on their app and £1 billion in annual revenue generated through digital channels, is embarking on a major 5-year data transformation strategy. … company-wide on-prem to Azure migration. What You’ll Be Doing Lead the cloud architecture transformation (Azure focus) Work hands-on with SQL, Azure, and data pipeline development Engage with stakeholders across the business – educate, influence, and lead Collaborate with BI, Data Engineering, and Digital Analytics teams Design and deliver scalable architecture for massive … data sets and high-volume user bases What You’ll Bring Minimum 2 years’ experience in architecture (cloud/data focused) Strong hands-on technical background (SQL, Azure essential) Experience in on-prem to cloud migration (Azure preferred) Ability to work autonomously, with strong stakeholder management Experience working with large data volumes and customer More ❯

Data Engineer Manchester £70,000 + benefits package Our client is a data-driven SaaS company who are looking for an experienced Data Engineer who has successfully scaled a company’s data maturity before and is eager to do it again. This isn’t a maintenance role. It’s a rare opportunity … to lead a large-scale data re-architecture project from the ground up. You’ll have the autonomy and trust to make key architectural decisions, shaping the technical foundation that powers our entire business—from product and analytics to customer intelligence and growth. If you’re motivated by high-impact work and the challenge of building a best … in-class, scalable data platform, we’d love to talk. Skills Needed 4–6+ years of professional experience as a Senior or Lead Data Engineer , including leadership on at least one major data re-architecture project Deep expertise in SQL and Python , applying data engineering best practices such as testing, version control More ❯

Data Engineer Manchester £70,000 + benefits package Our client is a data-driven SaaS company who are looking for an experienced Data Engineer who has successfully scaled a company’s data maturity before and is eager to do it again. This isn’t a maintenance role. It’s a rare opportunity … to lead a large-scale data re-architecture project from the ground up. You’ll have the autonomy and trust to make key architectural decisions, shaping the technical foundation that powers our entire business—from product and analytics to customer intelligence and growth. If you’re motivated by high-impact work and the challenge of building a best … in-class, scalable data platform, we’d love to talk. Skills Needed 4–6+ years of professional experience as a Senior or Lead Data Engineer , including leadership on at least one major data re-architecture project Deep expertise in SQL and Python , applying data engineering best practices such as testing, version control More ❯

Data Engineer Manchester £70,000 + benefits package Our client is a data-driven SaaS company who are looking for an experienced Data Engineer who has successfully scaled a company’s data maturity before and is eager to do it again. This isn’t a maintenance role. It’s a rare opportunity … to lead a large-scale data re-architecture project from the ground up. You’ll have the autonomy and trust to make key architectural decisions, shaping the technical foundation that powers our entire business—from product and analytics to customer intelligence and growth. If you’re motivated by high-impact work and the challenge of building a best … in-class, scalable data platform, we’d love to talk. Skills Needed 4–6+ years of professional experience as a Senior or Lead Data Engineer , including leadership on at least one major data re-architecture project Deep expertise in SQL and Python , applying data engineering best practices such as testing, version control More ❯

Data Delivery and Integration Manager (Remote) - Full time permanent role - Retail industry - Remote/home-based - Salary up to £75,000 plus bonus, private healthcare and more! A large and growing retail business are looking to expand their IT function with the addition of a Data Integration Manager. This role will focus on end to end … data engineering and integration solutions for a company wide modernisation programme. Duties will include: - Managing the processes for data engineering and data integration in order to align to business needs - Leading a team across engineering and integration - Work to ensure the data infrastructure of the business effectively supports analytics, AL, machine learning … ML and reporting - Designing of data pipelines, ETL, ELT processes and architecture - Work with internal stakeholders and multiple 3rd parties to seamlessly integrate systems, platforms and applications To be considered suitable for this Data Engineering Manager role you will need to have experience across the following: - Team leadership and guidance - Data engineering and aligning More ❯

Data Delivery and Integration Manager (Remote) - Full time permanent role - Retail industry - Remote/home-based - Salary up to £75,000 plus bonus, private healthcare and more! A large and growing retail business are looking to expand their IT function with the addition of a Data Integration Manager. This role will focus on end to end … data engineering and integration solutions for a company wide modernisation programme. Duties will include: - Managing the processes for data engineering and data integration in order to align to business needs - Leading a team across engineering and integration - Work to ensure the data infrastructure of the business effectively supports analytics, AL, machine learning … ML and reporting - Designing of data pipelines, ETL, ELT processes and architecture - Work with internal stakeholders and multiple 3rd parties to seamlessly integrate systems, platforms and applications To be considered suitable for this Data Engineering Manager role you will need to have experience across the following: - Team leadership and guidance - Data engineering and aligning More ❯

Senior Data Engineer in Databricks - Lead the Future of Data in Financial Services Location: Hybrid | 2-3 days/week on site Offices: Banbury, Milton Keynes, Bedford, Northampton, Manchester Salary: Up to £80,000 + Car Allowance + Bonus + Benefits Note: Visa sponsorship is not available for this role. Be the Technical Leader in a … Bold Data Transformation Our client is entering a pivotal phase in its data journey. With a new Head of Data already in place and a Data Architect joining shortly, the business is investing in a modern, scalable data platform to turn messy, complex client data into trusted, actionable … team. Finance sector experience is a bonus-but not essential. Interview Process Initial coffee chat to explore team fit and technical challenges. Technical interview covering Databricks, MDM, semantic layers, pipeline design, and governance. This is a rare opportunity to lead, build, and shape the future of data in a growing financial services business. Ready to make your More ❯