1 to 25 of 1,191 Data Pipeline Jobs in the UK

a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data … technical solutions. Communicate technical concepts effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote and implement DevOps best practices. … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data … technical solutions. Communicate technical concepts effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote and implement DevOps best practices. … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

Senior Data Engineer. Python, SQL, Alteryx, ETL, Power BI, Collibra. Energy Trading. £85,000-£90,000 + Bonus and Benefits. Hybrid 3 Days a week in Central London office. My client is a top tier energy trading firm based in Central London. They are looking for a permanent Senior Data Engineer. The Data Management … team is part of the Data Team which works closely with the Front Office and is the centre of data management, data quality and control and it has direct responsibility over the development and maintenance of EDF Trading’s datasets. Main responsibilities This role bridges hands on data engineering with data governance. You will design and operate robust data pipelines and models with a focus on embedding governance by design - capturing metadata and lineage, enforcing access controls and data quality, and ensuring our catalogue and stewardship model reflect the reality of how data flows across the firm. You will collaborate with traders, analysts More ❯

Engineer to join our Service Operations team. This role offers an exciting opportunity to learn, contribute to, and grow within the critical areas of data pipeline development, maintenance, and optimisation for our client-facing products and services. Working closely with experienced developers, data analysts, data scientists, and other stakeholders, you will … be instrumental in ensuring the efficient and reliable flow of data, building foundational skills in a supportive environment. Key Responsibilities Data Pipeline Development • Contribute to the design, building, and maintenance of scalable data pipelines • Implement ETL (Extract, Transform, Load) processes to ingest data from various sources • Ensure data quality … for data access and manipulation Documentation and Reporting: • Document data processes, pipelines, and architectures • Generate reports and visualisations to communicate data insights and pipeline performance Technical Support: • Provide technical support for data-related issues and troubleshoot problems as they arise • Assist in the implementation of data governance and compliance More ❯

The Opportunity Are you passionate about data engineering? The Open Data team at the Parliamentary Digital Service (PDS) is beginning the development of a new Open Data Platform for parliamentary procedure data, replacing existing platforms and their associated data integration and publishing services. As our Senior Data Engineer … you will play a key role in designing, building, and evolving the data platforms that support decision-making across the organisation. This role sits at the intersection of engineering, analytics, and product delivery. You'll work closely with cross-functional teams to provide high-quality, reliable, and scalable data solutions that drive insight and operational excellence. … experienced Data Engineer who will play a central role in building and supporting the new platform. The role will focus on developing the ETL data pipeline framework to integrate upstream master data from parliamentary business systems, as well as engineering new data services, including APIs and web-client applications. The Data More ❯

About The Role Job level: 10 Were looking for a Lead Data Engineer to join our Data Engineering and Analytics practice. In this role, you will: Lead the design, development, management and optimisation of data pipelines to ensure efficient data flows, recognising and sharing opportunities to reuse data flows where … possible. Coordinate teams and set best practices and standards when it comes to data engineering principles. Champion data engineering across projects and clients. Responsibilities Lead by example, holding responsibilities for team culture, and how projects deliver the most impact and value to our clients. Be accountable for the strategic direction, delivery and growth of our work. … understanding of data wareho using concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability to communicate technical concepts to both technical and non More ❯

Head of Data Engineering - Preston A Head of Data Engineering/Data Engineering Manager to lead the design, development, and enhancement of the data infrastructure and pipelines is required by a leading company based in Preston. The role offers Hybrid working, so 2 - 3 days in the office a week. You must … solutions for delivering high-quality care. The ideal candidate will lead the design and execution of cloud-based based scalable data storage solutions, oversee ETL pipeline development and optimisation, establish and manage data schemas and dictionaries, develop data integration solutions, and lead data cleansing, validation, and enrichment processes. Duties … You'll be on the lookout for performance bottlenecks and working to optimise data processing and query performance. This will help make sure your data pipeline is scalable and ready to handle big loads. You'll also be setting up monitoring frameworks to keep an eye on how your data pipeline is More ❯

the largest geospatial businesses in the UK. The new division provides a compelling capability that covers the full geospatial lifecycle, from strategy and consultancy to powerful software and data as a service. Geospatial data significantly underpins the division’s current and future revenues. Within the Division’s Data & Research Team, the Data Engineer role is responsible for designing, building and optimising our data pipelines and infrastructure, ensuring that the curation, delivery and value added use of our geospatial data catalogue is undertaken to a high quality. Working with the 3 sub teams of Research & Innovation, Data Delivery and Data Management you will … ensure we provide quality assured data products and services to our customers, maximising the potential for automation of our data pipelines, contributing to the development of new and existing data products and services and supporting our research and innovation projects. Continuous improvement is key to our success and you will directly contribute to the More ❯

Principal Data Architect (Data & Analytics) Excellent Salary, Bonus & Package Edinburgh (Hybrid, 3 days per week in the office) Permanent Posted Tue 28 Oct 2025 CVs ASAP Flex start date, Nov 25 – Apr 26 Meraki Talent are actively looking for a Principal Data Architect (Data & Analytics) to join their well-regarded global client … who are investing heavily in technology. The Principal Data Architect (Data & Analytics) will help deliver best practice data architecture and integrate data with their blue-chip client base. This person will own the design and delivery of data architecture whilst embracing the latest tech. This would be the perfect role … for someone who is keen to remain hands on and in touch with the business, whilst engaging with C Suite stakeholders and personally being responsible for data strategy. The Principal Data Architect (Data & Analytics) will the authority on data architecture across the organisation. Responsibilities of the Principal Data Architect (Data More ❯

Data Pipeline Engineer (On prem to AWS migration)UtilitiesPredominantly remote: occasional travel to Bristol6 months+£700 - £750 per day In short: We require 3 Data Engineers to join a large on prem to AWS migration programme in replicating data pipelines in readiness for a lift and shift to the cloud. Essential: AWS and … Redshift In full: Role purpose Reporting to the Lead Data Engineer, the Data Engineer is responsible for designing and maintaining scalable data pipelines, ensuring data availability, quality, and performance to support analytics and operational decision-making. They contribute to the data engineering roadmap through research, technical vision, business alignment, and … Ensure quality is maintained throughout the data engineering lifecycle, delivering robust, scalable, and cost-effective solutions on time and within budget. Monitor and drive data pipeline lifecycle, from design and implementation through to optimisation and post-deployment performance analysis. Support operational resilience, ensuring data solutions are maintainable, supportable, and aligned with architectural principles. More ❯

Data Pipeline Engineer (On prem to AWS migration) Utilities Predominantly remote: occasional travel to Bristol 6 months+ £700 - £750 per day In short: We require 3 Data Engineers to join a large on prem to AWS migration programme in replicating data pipelines in readiness for a lift and shift to the cloud. Essential … AWS and Redshift In full: Role purpose Reporting to the Lead Data Engineer, the Data Engineer is responsible for designing and maintaining scalable data pipelines, ensuring data availability, quality, and performance to support analytics and operational decision-making. They contribute to the data engineering roadmap through research, technical vision, business … Ensure quality is maintained throughout the data engineering lifecycle, delivering robust, scalable, and cost-effective solutions on time and within budget. Monitor and drive data pipeline lifecycle, from design and implementation through to optimisation and post-deployment performance analysis. Support operational resilience, ensuring data solutions are maintainable, supportable, and aligned with architectural principles. More ❯

Lead Data Engineer - Preston A Lead Data Engineer to lead the design, development, and enhancement of the data infrastructure and pipelines is required by a leading company based in Preston. The role offers Hybrid working, so 2 - 3 days in the office a week. You must have the following: Proven experience as a Senior … solutions for delivering high-quality care. The ideal candidate will lead the design and execution of cloud-based based scalable data storage solutions, oversee ETL pipeline development and optimisation, establish and manage data schemes and dictionaries, develop data integration solutions, lead data cleansing, validation, and enrichment processes. Duties include … You'll be on the lookout for performance bottlenecks and working to optimise data processing and query performance. This will help make sure your data pipeline is scalable and ready to handle big loads. Also be setting up monitoring frameworks to keep an eye on how your data pipeline is doing. Be More ❯

Junior Senior Big Data Developer (Python & SQL Focus) 📊 !!! URGENT HIRING !!! Looking for London based Citizens/Dependents ***We DO NOT provide SPONSORSHIP*** We're looking for an enthusiastic and detail-oriented Junior Big Data Developer to join our data engineering team. This role is ideal for an early-career professional with foundational knowledge in … data processing, strong proficiency in Python , and expert skills in SQL . You'll … focus on building, testing, and maintaining data pipelines and ensuring data quality across our scalable Big Data platforms. Key Responsibilities Data Pipeline Development: Assist in the design, construction, and maintenance of robust ETL/ELT pipelines to integrate data from various sources into our data warehouse or More ❯

Junior Senior Big Data Developer (Python & SQL Focus) 📊 !!! URGENT HIRING !!! Looking for London based Citizens/Dependents ***We DO NOT provide SPONSORSHIP*** We're looking for an enthusiastic and detail-oriented Junior Big Data Developer to join our data engineering team. This role is ideal for an early-career professional with foundational knowledge in … data processing, strong proficiency in Python , and expert skills in SQL . You'll … focus on building, testing, and maintaining data pipelines and ensuring data quality across our scalable Big Data platforms. Key Responsibilities Data Pipeline Development: Assist in the design, construction, and maintenance of robust ETL/ELT pipelines to integrate data from various sources into our data warehouse or More ❯

Role Title: Senior Data Engineer Department: IT and Business Change Reports To: Lead Stack Azure BI Developer Contract Type: Permanent Working Arrangements/Pattern: Full Time, 35 hours Working Location: FatFace HQ, Havant Who are we? For over 30 years, we’ve travelled, we’ve laughed, we’ve grown. We believe life is for living and more fun … environment free from prejudice, racism, sexism, harassment, bullying and any other form of marginalisation in our workplaces. Role Summary This role requires an experienced and technically skilled Senior Data Engineer to play a key role in our Data & Reporting team and drive the transformation of reporting services and data infrastructure, whilst acting as a … data into useful, actionable information. Key Contacts and Relationships: Data & Reporting Team : Collaborate closely with team members to support ETL processes, troubleshoot data pipeline issues, and improve data architecture. Internal Stakeholders : Work with various departments (e.g., finance, operations, marketing) to understand data requirements and provide fit-for-purpose analytical More ❯

supporting platform for front-office developers (quants and strategists) in a unique hands-on opportunity within a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with key internal platforms and front-office … problems, staying current with trends like AI Engineering, and want to make a difference in a globally respected financial institution, this role is for you. The successful AWS Data Engineer candidate will have the chance to make a significant impact in designing the platform and working on cutting-edge technologies like Databricks and … Snowflake in the heart of a leading global Investment Banks’ front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We’re Looking For 5 + years, hands-on experience More ❯

supporting platform for front-office developers (quants and strategists) in a unique hands-on opportunity within a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with key internal platforms and front-office … problems, staying current with trends like AI Engineering, and want to make a difference in a globally respected financial institution, this role is for you. The successful AWS Data Engineer candidate will have the chance to make a significant impact in designing the platform and working on cutting-edge technologies like Databricks and … Snowflake in the heart of a leading global Investment Banks’ front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We’re Looking For 5 + years, hands-on experience More ❯

strategists). This is a unique opportunity to remain hands-on in engineering while also leading a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with key internal platforms and front-office … staying current with trends like AI Engineering, and want to make a difference in a globally respected financial institution, this role is for you. The successful Senior AWS Data Engineer candidate will have the chance to make a significant impact in designing the platform and working on cutting-edge technologies like Databricks and … Snowflake in the heart of a leading global Investment Banks’ front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We’re Looking For 10 + years, hands-on experience More ❯

strategists). This is a unique opportunity to remain hands-on in engineering while also leading a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with key internal platforms and front-office … staying current with trends like AI Engineering, and want to make a difference in a globally respected financial institution, this role is for you. The successful Senior AWS Data Engineer candidate will have the chance to make a significant impact in designing the platform and working on cutting-edge technologies like Databricks and … Snowflake in the heart of a leading global Investment Banks’ front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We’re Looking For 10 + years, hands-on experience More ❯

Product Manager – Data & Data Science Remote (UK) Up to £90,000 + Benefits About the Company Our client is an award-winning search intelligence platform that helps global brands, marketers, and agencies gain an unparalleled understanding of the paid search landscape. Powered by patented AI technology, they enable clients to optimise spend, increase ROI, and stay … they’ve been redefining search intelligence for over a decade. In 2024, they secured multiple industry awards including Best AI Search Software Solution , Best Use of AI for Data , and Best PPC Management Software Suite . You’ll be joining a business that combines the maturity and stability of an established company with the agility, autonomy, and growth … a strong, people-first culture with a 93% approval rating and high employee satisfaction. About the Role You’ll join the Product function, working alongside leading AI and data specialists. The team is focused on delivering innovative, data-driven products that make a measurable difference to customers and the business. As the Product Manager for the More ❯

support EQ's transition into a stronger, more profitable, product-led business, driven by real insights and the needs of its customers. We are looking for a skilled Data Engineer to join our Data team. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data pipelines … using Microsoft Fabric and Databricks. You will work closely with our Product and Engineering teams to ensure data flow and integration across our data ecosystem. Your expertise will help us to improve our data infrastructure, improve data quality, and enable data-driven decision-making across the organization. Core Duties and … experience in data engineering, with a focus on cloud-based data pipelines and architectures Strong expertise in Microsoft Fabric and Databricks, including data pipeline development, data warehousing, and data lake management Proficiency in Python, SQL, Scala, or Java Experience with data processing frameworks such as Apache Spark More ❯

drive the accelerated digitalization of our talent. As the D&A Technical Lead for Smart Factories (Global), you will play a pivotal role in shaping and executing the Data Strategy for Smart Factory within Pet Nutrition Division. You will collaborate with cross-functional and cross-regional teams to develop, implement, and optimize Smart Factories strategies that align with … suitable alternative role is not available at that time, your employment will conclude. What will be your key responsibilities? Determine, implement and ensure adherence to global architectural and data standards and principles for Pet Nutrition Data & Analytics products across all regional markets, in conjunction with the Petcare Data Product leaders and Infrastructure teams. Create … such as: Azure Stack \ Data Bricks \ Python (inc. Django framework) Manufacturing data platform tools like Thingworx, Litmus is a plus Experience in data pipeline architecture. Knowledge of containerisation/microservices (Kubernetes/docker). Knowledge of API architecture REST, Webhooks, SOAP. Experience working in a multi-business unit or multi-client environment that More ❯

IR35) Databricks Engineer needed with active SC Security Clearance for 6 Month Contract based in Central London (Hybrid). Developing a cutting-edge Azure Databricks platform for economic data modelling, analysis, and forecasting. Start ASAP in Nov/Dec 2025. Hybrid Working - 2 days/week remote (WFH), and 3 days/week working on-site from the … Central London office. A chance to work with a leading global IT and Digital transformation business specialising in Government projects: In-depth Data Engineering + strong hands-on Azure Databricks expertise. Azure Data Services, Azure Data Factory, Azure Blob Storage + Azure SQL Database. Designing, developing, building + optimising data pipelines, implementing … including data modelling techniques + data integration patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data More ❯

IR35) Databricks Engineer needed with active SC Security Clearance for 6 Month Contract based in Central London (Hybrid). Developing a cutting-edge Azure Databricks platform for economic data modelling, analysis, and forecasting. Start ASAP in Nov/Dec 2025. Hybrid Working - 2 days/week remote (WFH), and 3 days/week working on-site from the … Central London office. A chance to work with a leading global IT and Digital transformation business specialising in Government projects: In-depth Data Engineering + strong hands-on Azure Databricks expertise. Azure Data Services, Azure Data Factory, Azure Blob Storage + Azure SQL Database. Designing, developing, building + optimising data pipelines, implementing … including data modelling techniques + data integration patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data More ❯

Position: Analytics Engineer Location: London (hybrid 3 times a week)Department: ITType: 3 months rolling contractOutside IR35 The client is seeking an experienced Analytics Engineer to join its Data & Analytics team. The role focuses on building scalable data pipelines, transforming raw data into clean, reusable datasets, and enabling data-driven decision-making. … Key Responsibilities Design and build data products, with proficiency throughout the data lifecycle. Develop robust data models through close collaboration with business users and the engineering team. Partner with senior management, operational leads, and other stakeholders, coaching and supporting a data-driven culture, including KPI definition and reporting frameworks. Accountable for data extraction, transforming JSON & XML, utlising high experience within metadata management. Collaborate with data engineers to develop data, enrich product design and integrate data for predictive models or machine learning. Deliver well-defined, transformed, tested, documented, and code-reviewed datasets for analysis. Evaluate and recommend improvements in data flow, influencing and More ❯