176 to 200 of 970 Data Pipeline Jobs in the UK excluding London

Technical Data Architect (Azure) Milton Keynes/WFH to £90k Are you a Data technologist with data architecture experience looking for an opportunity to take ownership and make an impact? You could be progressing your career at a well known household name within the fast food space as they invest heavily in their data strategy. As the Technical Data Architect you'll be instrumental in delivering a modern cloud based data infrastructure, they have 16 million customer data points across the UK and Ireland that they're looking to use this to optimise customer retention, product quality and speedy and efficient delivery to the customer. Initially … you'll be responsible for migrating the current on-premise data platform to Azure using Databricks; you'll collaborate with the Head of Digital and AI to modernise the current Data infrastructure and monetise the data available, working with senior business stakeholders to deliver tangible business value. Location/WFH: There's a hybrid More ❯

Data Engineer x4 days a week in their London office Join a fast-scaling, product-led tech company on a mission to transform how their sector performs in real time. With a strong focus on data-led decision-making and seamless mobile experiences, this team is building out their next-generation data infrastructure, and … they’re looking for another Data Engineer to help drive that forward. This is an exciting opportunity to shape the way data flows through a live product used by thousands, while working closely with analytics, engineering, and commercial teams. You’ll be joining a small, high-impact team, reporting to a technical leader with a strong … background in applied data architecture. What You’ll Be Doing Designing, building, and maintaining scalable data pipelines to power both internal analytics and customer-facing mobile/web applications Developing robust ETL processes for ingesting and transforming data from a variety of sources, particularly APIs Creating and optimising data architectures to ensure More ❯

Data Engineer x4 days a week in their London office Join a fast-scaling, product-led tech company on a mission to transform how their sector performs in real time. With a strong focus on data-led decision-making and seamless mobile experiences, this team is building out their next-generation data infrastructure, and … they’re looking for another Data Engineer to help drive that forward. This is an exciting opportunity to shape the way data flows through a live product used by thousands, while working closely with analytics, engineering, and commercial teams. You’ll be joining a small, high-impact team, reporting to a technical leader with a strong … background in applied data architecture. What You’ll Be Doing Designing, building, and maintaining scalable data pipelines to power both internal analytics and customer-facing mobile/web applications Developing robust ETL processes for ingesting and transforming data from a variety of sources, particularly APIs Creating and optimising data architectures to ensure More ❯

Azure Senior Data Lead with Azure Data Factory (ADF), Azure Databricks, SQL, Oracle PL/SQL, Python, Power BI. Job Summary The Senior Technical Lead will be responsible for leading the technical aspects of projects related to Azure Data Factory (ADF), Azure Databricks, SQL, Oracle PL/SQL, Power BI and Python. The role … involves designing, developing, and maintaining data pipelines and ETL processes while ensuring high performance and scalability of data solutions. (1.) Key Responsibilities 1. Lead the end-to-end technical implementation of data projects using azure data factory, azure databricks, sql, oracle pl/sql, Power BI and python. 2. Design and develop … efficient and reliable etl processes for large datasets. 3. Collaborate with cross functional teams to understand business requirements and translate them into technical solutions. 4. Optimize data workflows, troubleshoot issues, and ensure data quality and integrity. 5. Implement best practices for data security, governance, and compliance. 6. Provide technical guidance, mentoring, and support to More ❯

Azure Senior Data Lead with Azure Data Factory (ADF), Azure Databricks, SQL, Oracle PL/SQL, Python, Power BI. Job Summary The Senior Technical Lead will be responsible for leading the technical aspects of projects related to Azure Data Factory (ADF), Azure Databricks, SQL, Oracle PL/SQL, Power BI and Python. The role … involves designing, developing, and maintaining data pipelines and ETL processes while ensuring high performance and scalability of data solutions. (1.) Key Responsibilities 1. Lead the end-to-end technical implementation of data projects using azure data factory, azure databricks, sql, oracle pl/sql, Power BI and python. 2. Design and develop … efficient and reliable etl processes for large datasets. 3. Collaborate with cross functional teams to understand business requirements and translate them into technical solutions. 4. Optimize data workflows, troubleshoot issues, and ensure data quality and integrity. 5. Implement best practices for data security, governance, and compliance. 6. Provide technical guidance, mentoring, and support to More ❯

Data Engineer (Legal sector) Greater Manchester Hybrid (2 – 3 days per month in the office) £38,000 - £40,000 Only candidates within a commutable distance of Greater Manchester will be considered. About the Role Our legal client is seeking a Data Engineer to join their Data Team to design, build, and maintain an Azure … based data environment. Within this role you will architect and manage the data ecosystem, develop pipelines, oversee the data lake and enable high-performance reporting through Power BI. You’ll collaborate with Stakeholders to deliver the data roadmap, ensure a single source of truth, and help define a robust data strategy. This is a great opportunity to bring fresh ideas and shape the future of data within a growing organisation. Key Responsibilities Work with the Head of Department to prioritise and deliver data requirements into the data lake. Build, schedule, and manage data pipelines from operational systems into Azure. Maintain More ❯

Data Engineer £500 - £560 per day London - 1 day per week in office We're working with a leading global healthcare technology company who are building out their next-generation data platform, with a strong emphasis on automation, testing, and cloud-native engineering, and are looking for an experienced Data Engineer to join their … team. The Role You'll be part of a modern data engineering function that's implementing best-in-class data practices across ingestion, transformation, and orchestration layers. The environment is highly technical, collaborative, and fast-paced, giving you the opportunity to work across a wide variety of data sources and tools. Day-to-day … responsibilities include: Designing and developing DBT models and Airflow pipelines within a modern data stack. Building robust data ingestion pipelines across multiple sources - including external partners, internal platforms, and APIs. Implementing automated testing and CI/CD pipelines for data workflows. Performing data extraction and enrichment, including web scraping and parsing of More ❯

Data Engineer - FinTech - Newcastle (Tech stack: Data Engineer, SQL, Python, AWS, Git, Airflow, Data Pipelines, Data Platforms, Programmer, Developer, Architect, Data Engineer) Our client is a trailblazer in the FinTech space, known for delivering innovative technology solutions to global financial markets. They are expanding their engineering capability in Newcastle and … are looking for a talented Data Engineer to join their team. This role will focus on building and optimising systems that make complex datasets accessible, reliable, and valuable for the business. As a Data Engineer, you will take responsibility for the development of high-quality pipelines that process and manage large volumes of data from a range of external and internal sources. You'll play a key role in enhancing and maintaining their central data platform, ensuring the smooth delivery of information that supports investment decision-making. Working closely with stakeholders across the business, you'll help shape how data is accessed, tested, and leveraged to maximise value. More ❯

of the leading organizations in the UAE specializing in healthcare and advanced technology solutions . PLEASE NOTE THAT THE ROLE IS BASED IN DUBAI! As the Director – Healthcare Data and AI Solution Architect , you will play a key leadership role in defining and driving the company’s data and AI architecture strategy , shaping the foundation for … innovation, scalability, and excellence in healthcare technology. You will design and implement a robust, secure, and future-ready data and AI ecosystem , ensuring deep integration with enterprise-wide strategies and governance frameworks. Reporting to the Executive Director of AI & Data , you’ll lead a team of architects and collaborate closely with cross-functional teams to bring … the company’s AI vision to life. Key Responsibilities Architectural Leadership Define and maintain a comprehensive and scalable data and AI architecture aligned with enterprise-wide technology strategy. Establish and enforce best practices for data storage, integration, governance, and compliance with regulatory standards. Provide strategic technical direction to architecture and engineering teams. Evaluate and enhance systems More ❯

of the leading organizations in the UAE specializing in healthcare and advanced technology solutions . PLEASE NOTE THAT THE ROLE IS BASED IN DUBAI! As the Director – Healthcare Data and AI Solution Architect , you will play a key leadership role in defining and driving the company’s data and AI architecture strategy , shaping the foundation for … innovation, scalability, and excellence in healthcare technology. You will design and implement a robust, secure, and future-ready data and AI ecosystem , ensuring deep integration with enterprise-wide strategies and governance frameworks. Reporting to the Executive Director of AI & Data , you’ll lead a team of architects and collaborate closely with cross-functional teams to bring … the company’s AI vision to life. Key Responsibilities Architectural Leadership Define and maintain a comprehensive and scalable data and AI architecture aligned with enterprise-wide technology strategy. Establish and enforce best practices for data storage, integration, governance, and compliance with regulatory standards. Provide strategic technical direction to architecture and engineering teams. Evaluate and enhance systems More ❯

of the leading organizations in the UAE specializing in healthcare and advanced technology solutions . PLEASE NOTE THAT THE ROLE IS BASED IN DUBAI! As the Director – Healthcare Data and AI Solution Architect , you will play a key leadership role in defining and driving the company’s data and AI architecture strategy , shaping the foundation for … innovation, scalability, and excellence in healthcare technology. You will design and implement a robust, secure, and future-ready data and AI ecosystem , ensuring deep integration with enterprise-wide strategies and governance frameworks. Reporting to the Executive Director of AI & Data , you’ll lead a team of architects and collaborate closely with cross-functional teams to bring … the company’s AI vision to life. Key Responsibilities Architectural Leadership Define and maintain a comprehensive and scalable data and AI architecture aligned with enterprise-wide technology strategy. Establish and enforce best practices for data storage, integration, governance, and compliance with regulatory standards. Provide strategic technical direction to architecture and engineering teams. Evaluate and enhance systems More ❯

Senior Data Engineer – Cardiff/Hybrid – £65,000 - £75,000 + benefits Yolk Recruitment are proud to be supporting a leading global business investing heavily in its data and analytics capabilities. They’re looking for a Senior Data Engineer to help shape the next generation of their data platform — leading technical design … mentoring others, and driving best practice. This is a great opportunity for an experienced data professional who enjoys solving complex challenges, optimising large-scale systems, and influencing strategy within a collaborative, forward-thinking team. What you’ll be doing: Lead the design and implementation of scalable, high-performance data architectures and pipelines. Define and enforce best … practices for data engineering, including coding standards, testing, and documentation. Mentor and guide engineers, fostering collaboration and technical excellence. Translate complex business requirements into reliable, well-structured data solutions. Optimise data workflows for performance, reliability, and cost efficiency. Drive adoption of modern data tools and technologies across the organisation. Ensure robust data More ❯

Senior Data Engineer - Cardiff/Hybrid - £65,000 - £75,000 + benefits Yolk Recruitment are proud to be supporting a leading global business investing heavily in its data and analytics capabilities. They're looking for a Senior Data Engineer to help shape the next generation of their data platform - leading technical design … mentoring others, and driving best practice. This is a great opportunity for an experienced data professional who enjoys solving complex challenges, optimising large-scale systems, and influencing strategy within a collaborative, forward-thinking team. What you'll be doing: Lead the design and implementation of scalable, high-performance data architectures and pipelines. Define and enforce best … practices for data engineering, including coding standards, testing, and documentation. Mentor and guide engineers, fostering collaboration and technical excellence. Translate complex business requirements into reliable, well-structured data solutions. Optimise data workflows for performance, reliability, and cost efficiency. Drive adoption of modern data tools and technologies across the organisation. Ensure robust data More ❯

Data Engineer Hybrid (2 – 3 days per month in the office) Greater Manchester £38,000 - £40,000 DOE Only candidates within a commutable distance of Greater Manchester will be considered. About the Role Our legal client is seeking a Data Engineer to join their Data Team to design, build, and maintain an Azure-based … data environment. Within this role you will architect and manage the data ecosystem, develop pipelines, oversee the data lake and enable high-performance reporting through Power BI. You’ll collaborate with Stakeholders to deliver the data roadmap, ensure a single source of truth, and help define a robust data strategy. … This is a great opportunity to bring fresh ideas and shape the future of data within a growing organisation. Key Responsibilities Work with the Head of Department to prioritise and deliver data requirements into the data lake. Build, schedule, and manage data pipelines from operational systems into Azure. Maintain both production and More ❯

Data Engineer Hybrid (2 – 3 days per month in the office) Greater Manchester £38,000 - £40,000 DOE Only candidates within a commutable distance of Greater Manchester will be considered. About the Role Our legal client is seeking a Data Engineer to join their Data Team to design, build, and maintain an Azure-based … data environment. Within this role you will architect and manage the data ecosystem, develop pipelines, oversee the data lake and enable high-performance reporting through Power BI. You’ll collaborate with Stakeholders to deliver the data roadmap, ensure a single source of truth, and help define a robust data strategy. … This is a great opportunity to bring fresh ideas and shape the future of data within a growing organisation. Key Responsibilities Work with the Head of Department to prioritise and deliver data requirements into the data lake. Build, schedule, and manage data pipelines from operational systems into Azure. Maintain both production and More ❯

DV Cleared Data Engineers – Manchester – AWS or Azure – Senior/Principal We’re supporting a leading global consultancy with the expansion of their secure data engineering team, and we’re looking for DV Cleared Data Engineers to join major projects in Manchester. If you hold active DV clearance and want to work on cutting … edge, mission-critical programmes in a highly collaborative technical environment, this is an excellent opportunity. They are considering Data Engineers at all levels from Junior up to Principal Data Engineer. What You’ll Be Doing Designing and delivering secure, high-performance data pipelines and analytics platforms Working across AWS or Azure environments, building scalable … national infrastructure and security programmes with real impact What We’re Looking For Active DV clearance (essential) Strong background in data engineering, ETL, and data pipeline development Experience with AWS or Azure cloud technologies Excellent coding skills in Python, SQL, or Scala Understanding of data warehousing, modelling, and streaming technologies Confident working in More ❯

Lead/Senior Data Engineer Hybrid – Midlands (1-2 days per week in office) Salary: Up to £65k plus bonus Permanent We are partnering with a leading and innovative organisation to help recruit a Senior Data Engineer to join their evolving data team. This role provides a unique opportunity to work with cutting-edge … cloud data platforms, supporting the delivery of high-quality, reliable data solutions while contributing to automation and platform enhancements. What You’ll Be Doing As a Senior Data Engineer, you will play a key role in building, optimising, and maintaining cloud-based data solutions. Responsibilities include: Developing and maintaining end-to-end … data pipelines using Azure services such as Data Factory, Databricks, Synapse, and Data Lake. Designing and optimising data models, warehouses, and lakehouse architectures to support analytics and reporting requirements. Ensuring data governance, security, and compliance across cloud platforms, implementing access controls, encryption, and monitoring. Monitoring data processes, identifying More ❯

Lead/Senior Data Engineer Hybrid – Midlands (1-2 days per week in office) Salary: Up to £65k plus bonus Permanent We are partnering with a leading and innovative organisation to help recruit a Senior Data Engineer to join their evolving data team. This role provides a unique opportunity to work with cutting-edge … cloud data platforms, supporting the delivery of high-quality, reliable data solutions while contributing to automation and platform enhancements. What You’ll Be Doing As a Senior Data Engineer, you will play a key role in building, optimising, and maintaining cloud-based data solutions. Responsibilities include: Developing and maintaining end-to-end … data pipelines using Azure services such as Data Factory, Databricks, Synapse, and Data Lake. Designing and optimising data models, warehouses, and lakehouse architectures to support analytics and reporting requirements. Ensuring data governance, security, and compliance across cloud platforms, implementing access controls, encryption, and monitoring. Monitoring data processes, identifying More ❯

Data Engineer Hybrid (West Midlands) Permanent Competitive Salary + Benefits Are you passionate about building scalable data solutions and working with cutting-edge technologies like Microsoft Fabric, Azure, and Power BI? TXP is looking for a talented Data Engineer to join our growing team and help shape the future of data-driven … of technology and people, and we help everyone here to succeed. At TXP, you can multiply your potential. What You’ll Be Doing Designing, developing, and maintaining robust data pipelines. Building and optimising infrastructure for scalable data solutions. Integrating data from diverse sources across the business. Ensuring data integrity, security, and compliance … with best practices. Collaborating with cross-functional teams including analysts, developers, and business stakeholders. Leveraging Microsoft Fabric (Lakehouse, Data Pipelines) to modernise our data architecture. Supporting CI/CD and DevOps practices for data workflows. What We’re Looking For Proven experience as a Data Engineer or in a similar role. Strong More ❯

Data Engineer Hybrid (West Midlands) Permanent Competitive Salary + Benefits Are you passionate about building scalable data solutions and working with cutting-edge technologies like Microsoft Fabric, Azure, and Power BI? TXP is looking for a talented Data Engineer to join our growing team and help shape the future of data-driven … of technology and people, and we help everyone here to succeed. At TXP, you can multiply your potential. What You'll Be Doing Designing, developing, and maintaining robust data pipelines. Building and optimising infrastructure for scalable data solutions. Integrating data from diverse sources across the business. Ensuring data integrity, security, and compliance … with best practices. Collaborating with cross-functional teams including analysts, developers, and business stakeholders. Leveraging Microsoft Fabric (Lakehouse, Data Pipelines) to modernise our data architecture. Supporting CI/CD and DevOps practices for data workflows. What We're Looking For Proven experience as a Data Engineer or in a similar role. Strong More ❯

Senior Data Engineer Hybrid - Birmingham (2 days per week in office) Salary: Up to £65k Permanent We are partnering with a forward-thinking, public sector organisation currently undergoing a major data and digital transformation. They are seeking a Senior Data Engineer to play a pivotal role in developing and maintaining a modern, cloud-based … data platform that underpins critical regulatory and strategic objectives. This is an exciting opportunity to work with cutting-edge Microsoft technologies and contribute to the creation of a secure, scalable, and insight-driven data ecosystem that supports the organisation’s mission to deliver better outcomes for customers, communities, and the environment. What You’ll Be Doing … As a Senior Data Engineer, you’ll take a hands-on role in building and optimising data services across both modern and legacy platforms. Key responsibilities include: Developing, maintaining, and optimising data pipelines and ELT processes using Microsoft Fabric, Azure Data Factory, Synapse, Purview, and Power BI. Designing and implementing scalable and More ❯

Exp: 10 + yrs Job Summary The Senior Test Specialist/Architect in Cloud Automation Testing focuses on automating testing processes in ETL (Extract, Transform, Load) and Data Warehousing environments to ensure the quality and reliability of data pipelines. The role involves designing, developing, and implementing automation scripts for testing data transformations, data loading processes, and data quality verification. Key Responsibilities 1. Develop and execute automated test scripts for etl processes and data pipelines. 2. Collaborate with cross functional teams to design and implement automated testing frameworks. 3. Create test plans and test cases for etl and data warehouse testing. 4. Identify and troubleshoot issues … in data transformations and data loading processes. 5. Conduct performance testing and ensure scalability of data pipelines. 6. Implement best practices for data quality assurance in etl environments. Skill Requirements 1. Proficiency in cloud-based automation testing tools like selenium, appium, or similar. 2. Experience in automation testing of etl (extract, transform More ❯

Exp: 10 + yrs Job Summary The Senior Test Specialist/Architect in Cloud Automation Testing focuses on automating testing processes in ETL (Extract, Transform, Load) and Data Warehousing environments to ensure the quality and reliability of data pipelines. The role involves designing, developing, and implementing automation scripts for testing data transformations, data loading processes, and data quality verification. Key Responsibilities 1. Develop and execute automated test scripts for etl processes and data pipelines. 2. Collaborate with cross functional teams to design and implement automated testing frameworks. 3. Create test plans and test cases for etl and data warehouse testing. 4. Identify and troubleshoot issues … in data transformations and data loading processes. 5. Conduct performance testing and ensure scalability of data pipelines. 6. Implement best practices for data quality assurance in etl environments. Skill Requirements 1. Proficiency in cloud-based automation testing tools like selenium, appium, or similar. 2. Experience in automation testing of etl (extract, transform More ❯

Exp: 10 + yrs Job Summary The Senior Test Specialist/Architect in Cloud Automation Testing focuses on automating testing processes in ETL (Extract, Transform, Load) and Data Warehousing environments to ensure the quality and reliability of data pipelines. The role involves designing, developing, and implementing automation scripts for testing data transformations, data loading processes, and data quality verification. Key Responsibilities 1. Develop and execute automated test scripts for etl processes and data pipelines. 2. Collaborate with cross functional teams to design and implement automated testing frameworks. 3. Create test plans and test cases for etl and data warehouse testing. 4. Identify and troubleshoot issues … in data transformations and data loading processes. 5. Conduct performance testing and ensure scalability of data pipelines. 6. Implement best practices for data quality assurance in etl environments. Skill Requirements 1. Proficiency in cloud-based automation testing tools like selenium, appium, or similar. 2. Experience in automation testing of etl (extract, transform More ❯

Data EngineerLeicester£65,000 to £75,000Permanent I am currently looking for a Data Engineer to join a forward-thinking organisation, where you will play a pivotal role in building a modern, enterprise-scale data platform. Data Engineer - What will you be doing? * Designing and implementing modern data architectures including … lakehouse and medallion models using Azure and Databricks.* Developing, maintaining, and optimising data pipelines, quality validation tools, and lineage tracking systems to guarantee high-quality data outputs.* Manage MS SQL and Azure SQL databases efficiently while ensuring optimal performance across all environments.* Utilising Azure Data Factory, Python, and other tools to build efficient ETL … enterprise environments.* Power BI experience.* Deep understanding of SQL.* Experience with Azure and Databricks.* Proficient Python programming skills.* Demonstrated ability to design lakehouse architectures.* Experience implementing CI/CD pipeline management for reliable solution delivery. Robert Walters are proud to specialise in BI & Data recruitment across the UK, offering amazing opportunities on a permanent and interim basis. More ❯