1 to 25 of 1,177 Data Pipeline Jobs in the UK

a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data … technical solutions. Communicate technical concepts effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote and implement DevOps best practices. … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data … technical solutions. Communicate technical concepts effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote and implement DevOps best practices. … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

About The Role Job level: 10 Were looking for a Lead Data Engineer to join our Data Engineering and Analytics practice. In this role, you will: Lead the design, development, management and optimisation of data pipelines to ensure efficient data flows, recognising and sharing opportunities to reuse data flows where … possible. Coordinate teams and set best practices and standards when it comes to data engineering principles. Champion data engineering across projects and clients. Responsibilities Lead by example, holding responsibilities for team culture, and how projects deliver the most impact and value to our clients. Be accountable for the strategic direction, delivery and growth of our work. … understanding of data wareho using concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability to communicate technical concepts to both technical and non More ❯

Head of Data Engineering - Preston A Head of Data Engineering/Data Engineering Manager to lead the design, development, and enhancement of the data infrastructure and pipelines is required by a leading company based in Preston. The role offers Hybrid working, so 2 - 3 days in the office a week. You must … solutions for delivering high-quality care. The ideal candidate will lead the design and execution of cloud-based based scalable data storage solutions, oversee ETL pipeline development and optimisation, establish and manage data schemas and dictionaries, develop data integration solutions, and lead data cleansing, validation, and enrichment processes. Duties … You'll be on the lookout for performance bottlenecks and working to optimise data processing and query performance. This will help make sure your data pipeline is scalable and ready to handle big loads. You'll also be setting up monitoring frameworks to keep an eye on how your data pipeline is More ❯

the largest geospatial businesses in the UK. The new division provides a compelling capability that covers the full geospatial lifecycle, from strategy and consultancy to powerful software and data as a service. Geospatial data significantly underpins the division’s current and future revenues. Within the Division’s Data & Research Team, the Data Engineer role is responsible for designing, building and optimising our data pipelines and infrastructure, ensuring that the curation, delivery and value added use of our geospatial data catalogue is undertaken to a high quality. Working with the 3 sub teams of Research & Innovation, Data Delivery and Data Management you will … ensure we provide quality assured data products and services to our customers, maximising the potential for automation of our data pipelines, contributing to the development of new and existing data products and services and supporting our research and innovation projects. Continuous improvement is key to our success and you will directly contribute to the More ❯

Principal Data Architect (Data & Analytics) Excellent Salary, Bonus & Package Edinburgh (Hybrid, 3 days per week in the office) Permanent Posted Tue 28 Oct 2025 CVs ASAP Flex start date, Nov 25 – Apr 26 Meraki Talent are actively looking for a Principal Data Architect (Data & Analytics) to join their well-regarded global client … who are investing heavily in technology. The Principal Data Architect (Data & Analytics) will help deliver best practice data architecture and integrate data with their blue-chip client base. This person will own the design and delivery of data architecture whilst embracing the latest tech. This would be the perfect role … for someone who is keen to remain hands on and in touch with the business, whilst engaging with C Suite stakeholders and personally being responsible for data strategy. The Principal Data Architect (Data & Analytics) will the authority on data architecture across the organisation. Responsibilities of the Principal Data Architect (Data More ❯

Lead Data Engineer - Preston A Lead Data Engineer to lead the design, development, and enhancement of the data infrastructure and pipelines is required by a leading company based in Preston. The role offers Hybrid working, so 2 - 3 days in the office a week. You must have the following: Proven experience as a Senior … solutions for delivering high-quality care. The ideal candidate will lead the design and execution of cloud-based based scalable data storage solutions, oversee ETL pipeline development and optimisation, establish and manage data schemes and dictionaries, develop data integration solutions, lead data cleansing, validation, and enrichment processes. Duties include … You'll be on the lookout for performance bottlenecks and working to optimise data processing and query performance. This will help make sure your data pipeline is scalable and ready to handle big loads. Also be setting up monitoring frameworks to keep an eye on how your data pipeline is doing. Be More ❯

supporting platform for front-office developers (quants and strategists) in a unique hands-on opportunity within a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with key internal platforms and front-office … problems, staying current with trends like AI Engineering, and want to make a difference in a globally respected financial institution, this role is for you. The successful AWS Data Engineer candidate will have the chance to make a significant impact in designing the platform and working on cutting-edge technologies like Databricks and … Snowflake in the heart of a leading global Investment Banks’ front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We’re Looking For 5 + years, hands-on experience More ❯

supporting platform for front-office developers (quants and strategists) in a unique hands-on opportunity within a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with key internal platforms and front-office … problems, staying current with trends like AI Engineering, and want to make a difference in a globally respected financial institution, this role is for you. The successful AWS Data Engineer candidate will have the chance to make a significant impact in designing the platform and working on cutting-edge technologies like Databricks and … Snowflake in the heart of a leading global Investment Banks’ front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We’re Looking For 5 + years, hands-on experience More ❯

strategists). This is a unique opportunity to remain hands-on in engineering while also leading a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with key internal platforms and front-office … staying current with trends like AI Engineering, and want to make a difference in a globally respected financial institution, this role is for you. The successful Senior AWS Data Engineer candidate will have the chance to make a significant impact in designing the platform and working on cutting-edge technologies like Databricks and … Snowflake in the heart of a leading global Investment Banks’ front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We’re Looking For 10 + years, hands-on experience More ❯

strategists). This is a unique opportunity to remain hands-on in engineering while also leading a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with key internal platforms and front-office … staying current with trends like AI Engineering, and want to make a difference in a globally respected financial institution, this role is for you. The successful Senior AWS Data Engineer candidate will have the chance to make a significant impact in designing the platform and working on cutting-edge technologies like Databricks and … Snowflake in the heart of a leading global Investment Banks’ front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We’re Looking For 10 + years, hands-on experience More ❯

Product Manager – Data & Data Science Remote (UK) Up to £90,000 + Benefits About the Company Our client is an award-winning search intelligence platform that helps global brands, marketers, and agencies gain an unparalleled understanding of the paid search landscape. Powered by patented AI technology, they enable clients to optimise spend, increase ROI, and stay … they’ve been redefining search intelligence for over a decade. In 2024, they secured multiple industry awards including Best AI Search Software Solution , Best Use of AI for Data , and Best PPC Management Software Suite . You’ll be joining a business that combines the maturity and stability of an established company with the agility, autonomy, and growth … a strong, people-first culture with a 93% approval rating and high employee satisfaction. About the Role You’ll join the Product function, working alongside leading AI and data specialists. The team is focused on delivering innovative, data-driven products that make a measurable difference to customers and the business. As the Product Manager for the More ❯

support EQ's transition into a stronger, more profitable, product-led business, driven by real insights and the needs of its customers. We are looking for a skilled Data Engineer to join our Data team. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data pipelines … using Microsoft Fabric and Databricks. You will work closely with our Product and Engineering teams to ensure data flow and integration across our data ecosystem. Your expertise will help us to improve our data infrastructure, improve data quality, and enable data-driven decision-making across the organization. Core Duties and … experience in data engineering, with a focus on cloud-based data pipelines and architectures Strong expertise in Microsoft Fabric and Databricks, including data pipeline development, data warehousing, and data lake management Proficiency in Python, SQL, Scala, or Java Experience with data processing frameworks such as Apache Spark More ❯

drive the accelerated digitalization of our talent. As the D&A Technical Lead for Smart Factories (Global), you will play a pivotal role in shaping and executing the Data Strategy for Smart Factory within Pet Nutrition Division. You will collaborate with cross-functional and cross-regional teams to develop, implement, and optimize Smart Factories strategies that align with … suitable alternative role is not available at that time, your employment will conclude. What will be your key responsibilities? Determine, implement and ensure adherence to global architectural and data standards and principles for Pet Nutrition Data & Analytics products across all regional markets, in conjunction with the Petcare Data Product leaders and Infrastructure teams. Create … such as: Azure Stack \ Data Bricks \ Python (inc. Django framework) Manufacturing data platform tools like Thingworx, Litmus is a plus Experience in data pipeline architecture. Knowledge of containerisation/microservices (Kubernetes/docker). Knowledge of API architecture REST, Webhooks, SOAP. Experience working in a multi-business unit or multi-client environment that More ❯

IR35) Databricks Engineer needed with active SC Security Clearance for 6 Month Contract based in Central London (Hybrid). Developing a cutting-edge Azure Databricks platform for economic data modelling, analysis, and forecasting. Start ASAP in Nov/Dec 2025. Hybrid Working - 2 days/week remote (WFH), and 3 days/week working on-site from the … Central London office. A chance to work with a leading global IT and Digital transformation business specialising in Government projects: In-depth Data Engineering + strong hands-on Azure Databricks expertise. Azure Data Services, Azure Data Factory, Azure Blob Storage + Azure SQL Database. Designing, developing, building + optimising data pipelines, implementing … including data modelling techniques + data integration patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data More ❯

IR35) Databricks Engineer needed with active SC Security Clearance for 6 Month Contract based in Central London (Hybrid). Developing a cutting-edge Azure Databricks platform for economic data modelling, analysis, and forecasting. Start ASAP in Nov/Dec 2025. Hybrid Working - 2 days/week remote (WFH), and 3 days/week working on-site from the … Central London office. A chance to work with a leading global IT and Digital transformation business specialising in Government projects: In-depth Data Engineering + strong hands-on Azure Databricks expertise. Azure Data Services, Azure Data Factory, Azure Blob Storage + Azure SQL Database. Designing, developing, building + optimising data pipelines, implementing … including data modelling techniques + data integration patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data More ❯

Position: Analytics Engineer Location: London (hybrid 3 times a week)Department: ITType: 3 months rolling contractOutside IR35 The client is seeking an experienced Analytics Engineer to join its Data & Analytics team. The role focuses on building scalable data pipelines, transforming raw data into clean, reusable datasets, and enabling data-driven decision-making. … Key Responsibilities Design and build data products, with proficiency throughout the data lifecycle. Develop robust data models through close collaboration with business users and the engineering team. Partner with senior management, operational leads, and other stakeholders, coaching and supporting a data-driven culture, including KPI definition and reporting frameworks. Accountable for data extraction, transforming JSON & XML, utlising high experience within metadata management. Collaborate with data engineers to develop data, enrich product design and integrate data for predictive models or machine learning. Deliver well-defined, transformed, tested, documented, and code-reviewed datasets for analysis. Evaluate and recommend improvements in data flow, influencing and More ❯

Engineer Location: London (hybrid 3 times a week) Department: IT Type: 3 months rolling contract Outside IR35 The client is seeking an experienced Analytics Engineer to join its Data & Analytics team. The role focuses on building scalable data pipelines, transforming raw data into clean, reusable datasets, and enabling data-driven decision-making. … Key Responsibilities Design and build data products, with proficiency throughout the data lifecycle. Develop robust data models through close collaboration with business users and the engineering team. Partner with senior management, operational leads, and other stakeholders, coaching and supporting a data-driven culture, including KPI definition and reporting frameworks. Accountable for data extraction, transforming JSON & XML, utlising high experience within metadata management. Collaborate with data engineers to develop data, enrich product design and integrate data for predictive models or machine learning. Deliver well-defined, transformed, tested, documented, and code-reviewed datasets for analysis. Evaluate and recommend improvements in data flow, influencing and More ❯

The Platform & Engineering team at Royal Canin is responsible for managing our data capabilities. This includes the creation, operation, and optimisation of the data platform, assets, and pipelines. This is a growing team, supporting an advanced analytics agenda at Royal Canin that is rapidly transforming it into a business powered by data. To accelerate achieving this … objective, we are looking for enthusiastic data engineers to join our team. The role is responsible for creating and managing trusted Supply data assets. You will be seen as the primary engineering contact for this area, with an expectation to provide expert advice and technical leadership for product squads utilising data assets under your … models to support efficient and accurate analytical insight creation. Reduce data preparation efforts for solution users to expedite their processes and reduce errors. Perform data pipeline migrations if necessary. Implement alerting and monitoring capabilities to ensure high platform reliability in compliance with Mars Cyber Security Standards and Privacy Policies Ensure that product architecture and security More ❯

The Platform & Engineering team at Royal Canin is responsible for managing our data capabilities. This includes the creation, operation, and optimisation of the data platform, assets, and pipelines. This is a growing team, supporting an advanced analytics agenda at Royal Canin that is rapidly transforming it into a business powered by data. To accelerate achieving this … objective, we are looking for enthusiastic data engineers to join our team. The role is responsible for creating and managing trusted Supply data assets. You will be seen as the primary engineering contact for this area, with an expectation to provide expert advice and technical leadership for product squads utilising data assets under your … models to support efficient and accurate analytical insight creation. Reduce data preparation efforts for solution users to expedite their processes and reduce errors. Perform data pipeline migrations if necessary. Implement alerting and monitoring capabilities to ensure high platform reliability in compliance with Mars Cyber Security Standards and Privacy Policies Ensure that product architecture and security More ❯

and implementing robust, scalable, and efficient data systems that power analytics, machine learning models, and business insights. The ideal candidate will have expertise in data pipeline orchestration (e.g., Airflow), data lake and warehouse architecture and development, infrastructure as code (IaC) using Terraform , and data extraction from both structured and unstructured data … data models to support analytics and machine learning workflows. Design and implement scalable, secure, and high-performance data lake and data warehouse solutions. Pipeline Orchestration: Develop, monitor, and optimize ETL/ELT workflows using Apache Airflow. Ensure data pipelines are robust, error-tolerant, and scalable for real-time and batch processing. … Engineering, or a related field; or equivalent professional experience. Experience: 5+ years of experience in data engineering or a related field. Strong expertise in data pipeline orchestration tools such as Apache Airflow . Proven track record of designing and implementing data lakes and warehouses (experience with Azure is a plus). Demonstrated experience More ❯

DATA ARCHITECT - DV CLEARED NEW PERMANENT JOB OPPORTUNITY AVAILABLE WITHIN A LEADING NATIONAL SECURITY SME FOR A DATA ARCHITECT WITH ENHANCED DV CLEARANCE Permanent job opportunity for a Data Architect Leading National Security SME Salary up to £100,000 + Bonus London based organisation in an easily accessible location To apply please call or … email WHO WE ARE? We are recruiting multiple a Data Architect to support urgent National Security & Defence projects in London. Due to the nature of these projects you must hold enhanced DV Security Clearance. WHAT WILL THE DATA ARCHITECT BE DOING? You will be joining a leading SME who is working hard to support National Security … projects within UK Govt. Departments in London. As part of a team, you will be responsible for designing and implementing Data Solutions in Mission-Critical areas. WE NEED THE DATA ARCHITECT TO HAVE.... Current DV clearance - Enhanced Good at understanding complexity and abstracting that into a form that is consumable for a non-technical audience. Experience More ❯

Job summary The post-holder will provide technical expertise and leadership in data engineering, ensuring that data solutions are designed, developed, and deployed in line with NHS and organisational best practices. They will work within the Business Intelligence team to support the delivery of scalable, efficient, and innovative data solutions. Please note that previous … applicants need not apply. Main duties of the job The Lead Data & Systems Architect will oversee the development and implementation of cloud-based data pipelines, ensuring data quality, accessibility, and security. They will also support the Head of Data Engineering in defining best practices, supporting team development, and fostering a culture of … continuous improvement. The post-holder will collaborate with internal and external stakeholders to align data engineering solutions with national and local data strategies. They will ensure that data solutions support operational and analytical needs, contributing to effective decision-making and service improvements across the organisation. About us We are a multi award-winning organisation More ❯

Azure/Databricks Data Engineer x2 - UK Wide (Hybrid Working) £72,000-£80,000 Duties As a Databricks Data Engineer with an Azure focus, you will be an integral part of our team dedicated to building scalable and secure data platforms. You will leverage your expertise in Databricks, Apache Spark, and Azure to design … develop, and implement data warehouses, data lakehouses, and AI/ML models that fuel our data-driven operations. Skills/Experience Design and build high-performance data pipelines: Utilize Databricks and Apache Spark to extract, transform, and load data into Azure Data Lake Storage and other Azure services. … AI can support you in your delivery work Solid experience as a Data Engineer or similar role. Proven expertise in Databricks, Apache Spark, and data pipeline development and strong understanding of data warehousing concepts and practices. Experience with Microsoft Azure cloud platform, including Azure Data Lake Storage, Databricks and Azure Data More ❯

Billigence - Data Engineer Consultant About Billigence: Billigence is a boutique data consultancy with global outreach & clientele, transforming the way organizations work with data. We leverage proven, cutting-edge technologies to design, tailor, and implement advanced Business Intelligence solutions with high added value across a wide range of applications from process digitization through to Cloud Data Warehousing, Visualisation, Data Science, and Engineering or Data Governance. Headquartered in Sydney, Australia with offices around the world, we help clients navigate difficult business conditions, remove inefficiencies, and enable scalable adoption of analytics culture. About the Role: We are seeking a Consultant-Data Engineer to join our growing Practice. You will work … Understanding of data modelling principles, dimensional modelling, and database design Proficiency in SQL and query optimization Comprehensive knowledge of ETL/ELT processes and data pipeline architecture Excellent communication skills with the ability to collaborate across cross-functional teams Experience managing client relationships at various levels Strong problem-solving abilities and analytical thinking Highly Desirable More ❯