1 to 25 of 1,120 Data Pipeline Jobs in the UK

a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data … technical solutions. Communicate technical concepts effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote and implement DevOps best practices. … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets . Key Responsibilities: Data Pipeline Development & Optimisation: Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data … technical solutions. Communicate technical concepts effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote and implement DevOps best practices … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

About The Role Job level: 10 Were looking for a Lead Data Engineer to join our Data Engineering and Analytics practice. In this role, you will: Lead the design, development, management and optimisation of data pipelines to ensure efficient data flows, recognising and sharing opportunities to reuse data flows where … possible. Coordinate teams and set best practices and standards when it comes to data engineering principles. Champion data engineering across projects and clients. Responsibilities Lead by example, holding responsibilities for team culture, and how projects deliver the most impact and value to our clients. Be accountable for the strategic direction, delivery and growth of our work. … understanding of data wareho using concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability to communicate technical concepts to both technical and non More ❯

Head of Data Engineering - Preston A Head of Data Engineering/Data Engineering Manager to lead the design, development, and enhancement of the data infrastructure and pipelines is required by a leading company based in Preston. The role offers Hybrid working, so 2 - 3 days in the office a week. You must … solutions for delivering high-quality care. The ideal candidate will lead the design and execution of cloud-based based scalable data storage solutions, oversee ETL pipeline development and optimisation, establish and manage data schemas and dictionaries, develop data integration solutions, and lead data cleansing, validation, and enrichment processes. Duties … You'll be on the lookout for performance bottlenecks and working to optimise data processing and query performance. This will help make sure your data pipeline is scalable and ready to handle big loads. You'll also be setting up monitoring frameworks to keep an eye on how your data pipeline is More ❯

the largest geospatial businesses in the UK. The new division provides a compelling capability that covers the full geospatial lifecycle, from strategy and consultancy to powerful software and data as a service. Geospatial data significantly underpins the division’s current and future revenues. Within the Division’s Data & Research Team, the Data Engineer role is responsible for designing, building and optimising our data pipelines and infrastructure, ensuring that the curation, delivery and value added use of our geospatial data catalogue is undertaken to a high quality. Working with the 3 sub teams of Research & Innovation, Data Delivery and Data Management you will … ensure we provide quality assured data products and services to our customers, maximising the potential for automation of our data pipelines, contributing to the development of new and existing data products and services and supporting our research and innovation projects. Continuous improvement is key to our success and you will directly contribute to the More ❯

Principal Data Architect (Data & Analytics) Excellent Salary, Bonus & Package Edinburgh (Hybrid, 3 days per week in the office) Permanent Posted Tue 28 Oct 2025 CVs ASAP Flex start date, Nov 25 – Apr 26 Meraki Talent are actively looking for a Principal Data Architect (Data & Analytics) to join their well-regarded global client … who are investing heavily in technology. The Principal Data Architect (Data & Analytics) will help deliver best practice data architecture and integrate data with their blue-chip client base. This person will own the design and delivery of data architecture whilst embracing the latest tech. This would be the perfect role … for someone who is keen to remain hands on and in touch with the business, whilst engaging with C Suite stakeholders and personally being responsible for data strategy. The Principal Data Architect (Data & Analytics) will the authority on data architecture across the organisation. Responsibilities of the Principal Data Architect (Data More ❯

Lead Data Engineer - Preston A Lead Data Engineer to lead the design, development, and enhancement of the data infrastructure and pipelines is required by a leading company based in Preston. The role offers Hybrid working, so 2 - 3 days in the office a week. You must have the following: Proven experience as a Senior … solutions for delivering high-quality care. The ideal candidate will lead the design and execution of cloud-based based scalable data storage solutions, oversee ETL pipeline development and optimisation, establish and manage data schemes and dictionaries, develop data integration solutions, lead data cleansing, validation, and enrichment processes. Duties include … You'll be on the lookout for performance bottlenecks and working to optimise data processing and query performance. This will help make sure your data pipeline is scalable and ready to handle big loads. Also be setting up monitoring frameworks to keep an eye on how your data pipeline is doing. Be More ❯

supporting platform for front-office developers (quants and strategists) in a unique hands-on opportunity within a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with key internal platforms and front-office … problems, staying current with trends like AI Engineering, and want to make a difference in a globally respected financial institution, this role is for you. The successful AWS Data Engineer candidate will have the chance to make a significant impact in designing the platform and working on cutting-edge technologies like Databricks and … Snowflake in the heart of a leading global Investment Banks’ front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We’re Looking For 5 + years, hands-on experience More ❯

supporting platform for front-office developers (quants and strategists) in a unique hands-on opportunity within a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with key internal platforms and front-office … problems, staying current with trends like AI Engineering, and want to make a difference in a globally respected financial institution, this role is for you. The successful AWS Data Engineer candidate will have the chance to make a significant impact in designing the platform and working on cutting-edge technologies like Databricks and … Snowflake in the heart of a leading global Investment Banks’ front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We’re Looking For 5 + years, hands-on experience More ❯

strategists). This is a unique opportunity to remain hands-on in engineering while also leading a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with key internal platforms and front-office … staying current with trends like AI Engineering, and want to make a difference in a globally respected financial institution, this role is for you. The successful Senior AWS Data Engineer candidate will have the chance to make a significant impact in designing the platform and working on cutting-edge technologies like Databricks and … Snowflake in the heart of a leading global Investment Banks’ front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We’re Looking For 10 + years, hands-on experience More ❯

strategists). This is a unique opportunity to remain hands-on in engineering while also leading a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with key internal platforms and front-office … staying current with trends like AI Engineering, and want to make a difference in a globally respected financial institution, this role is for you. The successful Senior AWS Data Engineer candidate will have the chance to make a significant impact in designing the platform and working on cutting-edge technologies like Databricks and … Snowflake in the heart of a leading global Investment Banks’ front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We’re Looking For 10 + years, hands-on experience More ❯

Product Manager – Data & Data Science Remote (UK) Up to £90,000 + Benefits About the Company Our client is an award-winning search intelligence platform that helps global brands, marketers, and agencies gain an unparalleled understanding of the paid search landscape. Powered by patented AI technology, they enable clients to optimise spend, increase ROI, and stay … they’ve been redefining search intelligence for over a decade. In 2024, they secured multiple industry awards including Best AI Search Software Solution , Best Use of AI for Data , and Best PPC Management Software Suite . You’ll be joining a business that combines the maturity and stability of an established company with the agility, autonomy, and growth … a strong, people-first culture with a 93% approval rating and high employee satisfaction. About the Role You’ll join the Product function, working alongside leading AI and data specialists. The team is focused on delivering innovative, data-driven products that make a measurable difference to customers and the business. As the Product Manager for the More ❯

support EQ's transition into a stronger, more profitable, product-led business, driven by real insights and the needs of its customers. We are looking for a skilled Data Engineer to join our Data team. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data pipelines … using Microsoft Fabric and Databricks. You will work closely with our Product and Engineering teams to ensure data flow and integration across our data ecosystem. Your expertise will help us to improve our data infrastructure, improve data quality, and enable data-driven decision-making across the organization. Core Duties and … experience in data engineering, with a focus on cloud-based data pipelines and architectures Strong expertise in Microsoft Fabric and Databricks, including data pipeline development, data warehousing, and data lake management Proficiency in Python, SQL, Scala, or Java Experience with data processing frameworks such as Apache Spark More ❯

drive the accelerated digitalization of our talent. As the D&A Technical Lead for Smart Factories (Global), you will play a pivotal role in shaping and executing the Data Strategy for Smart Factory within Pet Nutrition Division. You will collaborate with cross-functional and cross-regional teams to develop, implement, and optimize Smart Factories strategies that align with … promotional opportunity with Mars before December 31, 2026. What are we looking for? 10+ years' experience working in this space, with particular focus on working with Manufacturing, CPG data At least 5 years' experience in a senior data/architecture/leadership role with demonstrable thought leadership abilities Experience in Architecture as a code approach. Experience … such as: Azure Stack, Data Bricks, Python (inc. Django framework) Manufacturing data platform tools like Thingworx, Litmus is a plus Experience in data pipeline architecture. Knowledge of containerisation/microservices (Kubernetes/docker). Knowledge of API architecture REST, Webhooks, SOAP. Experience working in a multi-business unit or multi-client environment that More ❯

drive the accelerated digitalization of our talent. As the D&A Technical Lead for Smart Factories (Global), you will play a pivotal role in shaping and executing the Data Strategy for Smart Factory within Pet Nutrition Division. You will collaborate with cross-functional and cross-regional teams to develop, implement, and optimize Smart Factories strategies that align with … suitable alternative role is not available at that time, your employment will conclude. What will be your key responsibilities? Determine, implement and ensure adherence to global architectural and data standards and principles for Pet Nutrition Data & Analytics products across all regional markets, in conjunction with the Petcare Data Product leaders and Infrastructure teams. Create … such as: Azure Stack \ Data Bricks \ Python (inc. Django framework) Manufacturing data platform tools like Thingworx, Litmus is a plus Experience in data pipeline architecture. Knowledge of containerisation/microservices (Kubernetes/docker). Knowledge of API architecture REST, Webhooks, SOAP. Experience working in a multi-business unit or multi-client environment that More ❯

IR35) Databricks Engineer needed with active SC Security Clearance for 6 Month Contract based in Central London (Hybrid). Developing a cutting-edge Azure Databricks platform for economic data modelling, analysis, and forecasting. Start ASAP in Nov/Dec 2025. Hybrid Working - 2 days/week remote (WFH), and 3 days/week working on-site from the … Central London office. A chance to work with a leading global IT and Digital transformation business specialising in Government projects: In-depth Data Engineering + strong hands-on Azure Databricks expertise. Azure Data Services, Azure Data Factory, Azure Blob Storage + Azure SQL Database. Designing, developing, building + optimising data pipelines, implementing … including data modelling techniques + data integration patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data More ❯

Position: Analytics Engineer Location: London (hybrid 3 times a week)Department: ITType: 3 months rolling contractOutside IR35 The client is seeking an experienced Analytics Engineer to join its Data & Analytics team. The role focuses on building scalable data pipelines, transforming raw data into clean, reusable datasets, and enabling data-driven decision-making. … Key Responsibilities Design and build data products, with proficiency throughout the data lifecycle. Develop robust data models through close collaboration with business users and the engineering team. Partner with senior management, operational leads, and other stakeholders, coaching and supporting a data-driven culture, including KPI definition and reporting frameworks. Accountable for data extraction, transforming JSON & XML, utlising high experience within metadata management. Collaborate with data engineers to develop data, enrich product design and integrate data for predictive models or machine learning. Deliver well-defined, transformed, tested, documented, and code-reviewed datasets for analysis. Evaluate and recommend improvements in data flow, influencing and More ❯

Engineer Location: London (hybrid 3 times a week) Department: IT Type: 3 months rolling contract Outside IR35 The client is seeking an experienced Analytics Engineer to join its Data & Analytics team. The role focuses on building scalable data pipelines, transforming raw data into clean, reusable datasets, and enabling data-driven decision-making. … Key Responsibilities Design and build data products, with proficiency throughout the data lifecycle. Develop robust data models through close collaboration with business users and the engineering team. Partner with senior management, operational leads, and other stakeholders, coaching and supporting a data-driven culture, including KPI definition and reporting frameworks. Accountable for data extraction, transforming JSON & XML, utlising high experience within metadata management. Collaborate with data engineers to develop data, enrich product design and integrate data for predictive models or machine learning. Deliver well-defined, transformed, tested, documented, and code-reviewed datasets for analysis. Evaluate and recommend improvements in data flow, influencing and More ❯

The Platform & Engineering team at Royal Canin is responsible for managing our data capabilities. This includes the creation, operation, and optimisation of the data platform, assets, and pipelines. This is a growing team, supporting an advanced analytics agenda at Royal Canin that is rapidly transforming it into a business powered by data. To accelerate achieving this … objective, we are looking for enthusiastic data engineers to join our team. The role is responsible for creating and managing trusted Supply data assets. You will be seen as the primary engineering contact for this area, with an expectation to provide expert advice and technical leadership for product squads utilising data assets under your … models to support efficient and accurate analytical insight creation. Reduce data preparation efforts for solution users to expedite their processes and reduce errors. Perform data pipeline migrations if necessary. Implement alerting and monitoring capabilities to ensure high platform reliability in compliance with Mars Cyber Security Standards and Privacy Policies Ensure that product architecture and security More ❯

The Platform & Engineering team at Royal Canin is responsible for managing our data capabilities. This includes the creation, operation, and optimisation of the data platform, assets, and pipelines. This is a growing team, supporting an advanced analytics agenda at Royal Canin that is rapidly transforming it into a business powered by data. To accelerate achieving this … objective, we are looking for enthusiastic data engineers to join our team. The role is responsible for creating and managing trusted Supply data assets. You will be seen as the primary engineering contact for this area, with an expectation to provide expert advice and technical leadership for product squads utilising data assets under your … models to support efficient and accurate analytical insight creation. Reduce data preparation efforts for solution users to expedite their processes and reduce errors. Perform data pipeline migrations if necessary. Implement alerting and monitoring capabilities to ensure high platform reliability in compliance with Mars Cyber Security Standards and Privacy Policies Ensure that product architecture and security More ❯

through an umbrella company Requirements 'must have': Education: Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent experience). 4+ years of experience developing data pipelines and data warehousing solutions using Python and libraries such as Pandas, NumPy, PySpark, etc. 3+ years hands-on experience with cloud services, especially Databricks, for building … within a dynamically evolving technical stack. Collaborate seamlessly across diverse technical stacks, including Databricks, Snowflake, etc. Developing various components in Python as part of a unified data pipeline framework. Contributing towards the establishment of best practices for the optimal and efficient usage of data across various on-prem and cloud platforms. Assisting with the testing … and deployment of our data pipeline framework utilizing standard testing frameworks and CI/CD tooling. Monitoring the performance of queries and data loads and perform tuning as necessary. Providing assistance and guidance during QA & UAT phases to quickly confirm the validity of potential issues and to determine the root cause and best resolution of More ❯

and implementing robust, scalable, and efficient data systems that power analytics, machine learning models, and business insights. The ideal candidate will have expertise in data pipeline orchestration (e.g., Airflow), data lake and warehouse architecture and development, infrastructure as code (IaC) using Terraform , and data extraction from both structured and unstructured data … data models to support analytics and machine learning workflows. Design and implement scalable, secure, and high-performance data lake and data warehouse solutions. Pipeline Orchestration: Develop, monitor, and optimize ETL/ELT workflows using Apache Airflow. Ensure data pipelines are robust, error-tolerant, and scalable for real-time and batch processing. … Engineering, or a related field; or equivalent professional experience. Experience: 5+ years of experience in data engineering or a related field. Strong expertise in data pipeline orchestration tools such as Apache Airflow . Proven track record of designing and implementing data lakes and warehouses (experience with Azure is a plus). Demonstrated experience More ❯

Location - London, Bristol or Manchester (1 day a month onsite) Duration - 6 months Rate - £550 - £600pd (inside ir35) As a Data Engineer in the Cyber and Domains Protection Team you will: Work within an Agile team to support the development of dashboards and build automated reports to meet the needs of technical and non-technical users Work with … the data analyst and user researcher to update relevant data models to allow business intelligence data to meet the organisation's specific needs Develop business intelligence reports that can be automated, reused and shared with users directly Implement data flows to connect operational systems, data for analytics and business intelligence … for ETL and data cataloging Amazon Redshift or Athena for data warehousing and analytics Lambda for event-driven data processing. ETL/ELT pipeline development : experience in designing, building, and maintaining robust, automated data pipelines. You should be comfortable with both the theory and practical application of extracting, transforming, and loading More ❯

Location - London, Bristol or Manchester (1 day a month onsite) Duration - 6 months Rate - £550 - £600pd (inside ir35) As a Data Engineer in the Cyber and Domains Protection Team you will: Work within an Agile team to support the development of dashboards and build automated reports to meet the needs of technical and non-technical users Work with … the data analyst and user researcher to update relevant data models to allow business intelligence data to meet the organisation's specific needs Develop business intelligence reports that can be automated, reused and shared with users directly Implement data flows to connect operational systems, data for analytics and business intelligence … for ETL and data cataloging Amazon Redshift or Athena for data warehousing and analytics Lambda for event-driven data processing. ETL/ELT pipeline development : experience in designing, building, and maintaining robust, automated data pipelines. You should be comfortable with both the theory and practical application of extracting, transforming, and loading More ❯

DATA ARCHITECT - DV CLEARED NEW PERMANENT JOB OPPORTUNITY AVAILABLE WITHIN A LEADING NATIONAL SECURITY SME FOR A DATA ARCHITECT WITH ENHANCED DV CLEARANCE Permanent job opportunity for a Data Architect Leading National Security SME Salary up to £100,000 + Bonus London based organisation in an easily accessible location To apply please call or … email WHO WE ARE? We are recruiting multiple a Data Architect to support urgent National Security & Defence projects in London. Due to the nature of these projects you must hold enhanced DV Security Clearance. WHAT WILL THE DATA ARCHITECT BE DOING? You will be joining a leading SME who is working hard to support National Security … projects within UK Govt. Departments in London. As part of a team, you will be responsible for designing and implementing Data Solutions in Mission-Critical areas. WE NEED THE DATA ARCHITECT TO HAVE.... Current DV clearance - Enhanced Good at understanding complexity and abstracting that into a form that is consumable for a non-technical audience. Experience More ❯