1 to 25 of 991 Data Pipeline Jobs in England

a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data … technical solutions. Communicate technical concepts effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote and implement DevOps best practices. … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data … technical solutions. Communicate technical concepts effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote and implement DevOps best practices. … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

The Opportunity Are you passionate about data engineering? The Open Data team at the Parliamentary Digital Service (PDS) is beginning the development of a new Open Data Platform for parliamentary procedure data, replacing existing platforms and their associated data integration and publishing services. As our Senior Data Engineer … you will play a key role in designing, building, and evolving the data platforms that support decision-making across the organisation. This role sits at the intersection of engineering, analytics, and product delivery. You'll work closely with cross-functional teams to provide high-quality, reliable, and scalable data solutions that drive insight and operational excellence. … experienced Data Engineer who will play a central role in building and supporting the new platform. The role will focus on developing the ETL data pipeline framework to integrate upstream master data from parliamentary business systems, as well as engineering new data services, including APIs and web-client applications. The Data More ❯

About The Role Job level: 10 Were looking for a Lead Data Engineer to join our Data Engineering and Analytics practice. In this role, you will: Lead the design, development, management and optimisation of data pipelines to ensure efficient data flows, recognising and sharing opportunities to reuse data flows where … possible. Coordinate teams and set best practices and standards when it comes to data engineering principles. Champion data engineering across projects and clients. Responsibilities Lead by example, holding responsibilities for team culture, and how projects deliver the most impact and value to our clients. Be accountable for the strategic direction, delivery and growth of our work. … understanding of data wareho using concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability to communicate technical concepts to both technical and non More ❯

Head of Data Engineering - Preston A Head of Data Engineering/Data Engineering Manager to lead the design, development, and enhancement of the data infrastructure and pipelines is required by a leading company based in Preston. The role offers Hybrid working, so 2 - 3 days in the office a week. You must … solutions for delivering high-quality care. The ideal candidate will lead the design and execution of cloud-based based scalable data storage solutions, oversee ETL pipeline development and optimisation, establish and manage data schemas and dictionaries, develop data integration solutions, and lead data cleansing, validation, and enrichment processes. Duties … You'll be on the lookout for performance bottlenecks and working to optimise data processing and query performance. This will help make sure your data pipeline is scalable and ready to handle big loads. You'll also be setting up monitoring frameworks to keep an eye on how your data pipeline is More ❯

Lead Data Engineer - Preston A Lead Data Engineer to lead the design, development, and enhancement of the data infrastructure and pipelines is required by a leading company based in Preston. The role offers Hybrid working, so 2 - 3 days in the office a week. You must have the following: Proven experience as a Senior … solutions for delivering high-quality care. The ideal candidate will lead the design and execution of cloud-based based scalable data storage solutions, oversee ETL pipeline development and optimisation, establish and manage data schemes and dictionaries, develop data integration solutions, lead data cleansing, validation, and enrichment processes. Duties include … You'll be on the lookout for performance bottlenecks and working to optimise data processing and query performance. This will help make sure your data pipeline is scalable and ready to handle big loads. Also be setting up monitoring frameworks to keep an eye on how your data pipeline is doing. Be More ❯

supporting platform for front-office developers (quants and strategists) in a unique hands-on opportunity within a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with key internal platforms and front-office … problems, staying current with trends like AI Engineering, and want to make a difference in a globally respected financial institution, this role is for you. The successful AWS Data Engineer candidate will have the chance to make a significant impact in designing the platform and working on cutting-edge technologies like Databricks and … Snowflake in the heart of a leading global Investment Banks’ front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We’re Looking For 5 + years, hands-on experience More ❯

supporting platform for front-office developers (quants and strategists) in a unique hands-on opportunity within a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with key internal platforms and front-office … problems, staying current with trends like AI Engineering, and want to make a difference in a globally respected financial institution, this role is for you. The successful AWS Data Engineer candidate will have the chance to make a significant impact in designing the platform and working on cutting-edge technologies like Databricks and … Snowflake in the heart of a leading global Investment Banks’ front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We’re Looking For 5 + years, hands-on experience More ❯

strategists). This is a unique opportunity to remain hands-on in engineering while also leading a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with key internal platforms and front-office … staying current with trends like AI Engineering, and want to make a difference in a globally respected financial institution, this role is for you. The successful Senior AWS Data Engineer candidate will have the chance to make a significant impact in designing the platform and working on cutting-edge technologies like Databricks and … Snowflake in the heart of a leading global Investment Banks’ front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We’re Looking For 10 + years, hands-on experience More ❯

strategists). This is a unique opportunity to remain hands-on in engineering while also leading a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with key internal platforms and front-office … staying current with trends like AI Engineering, and want to make a difference in a globally respected financial institution, this role is for you. The successful Senior AWS Data Engineer candidate will have the chance to make a significant impact in designing the platform and working on cutting-edge technologies like Databricks and … Snowflake in the heart of a leading global Investment Banks’ front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We’re Looking For 10 + years, hands-on experience More ❯

support EQ's transition into a stronger, more profitable, product-led business, driven by real insights and the needs of its customers. We are looking for a skilled Data Engineer to join our Data team. As a Data Engineer, you will be responsible for designing, building, and maintaining large-scale data pipelines … using Microsoft Fabric and Databricks. You will work closely with our Product and Engineering teams to ensure data flow and integration across our data ecosystem. Your expertise will help us to improve our data infrastructure, improve data quality, and enable data-driven decision-making across the organization. Core Duties and … experience in data engineering, with a focus on cloud-based data pipelines and architectures Strong expertise in Microsoft Fabric and Databricks, including data pipeline development, data warehousing, and data lake management Proficiency in Python, SQL, Scala, or Java Experience with data processing frameworks such as Apache Spark More ❯

drive the accelerated digitalization of our talent. As the D&A Technical Lead for Smart Factories (Global), you will play a pivotal role in shaping and executing the Data Strategy for Smart Factory within Pet Nutrition Division. You will collaborate with cross-functional and cross-regional teams to develop, implement, and optimize Smart Factories strategies that align with … suitable alternative role is not available at that time, your employment will conclude. What will be your key responsibilities? Determine, implement and ensure adherence to global architectural and data standards and principles for Pet Nutrition Data & Analytics products across all regional markets, in conjunction with the Petcare Data Product leaders and Infrastructure teams. Create … such as: Azure Stack \ Data Bricks \ Python (inc. Django framework) Manufacturing data platform tools like Thingworx, Litmus is a plus Experience in data pipeline architecture. Knowledge of containerisation/microservices (Kubernetes/docker). Knowledge of API architecture REST, Webhooks, SOAP. Experience working in a multi-business unit or multi-client environment that More ❯

IR35) Databricks Engineer needed with active SC Security Clearance for 6 Month Contract based in Central London (Hybrid). Developing a cutting-edge Azure Databricks platform for economic data modelling, analysis, and forecasting. Start ASAP in Nov/Dec 2025. Hybrid Working - 2 days/week remote (WFH), and 3 days/week working on-site from the … Central London office. A chance to work with a leading global IT and Digital transformation business specialising in Government projects: In-depth Data Engineering + strong hands-on Azure Databricks expertise. Azure Data Services, Azure Data Factory, Azure Blob Storage + Azure SQL Database. Designing, developing, building + optimising data pipelines, implementing … including data modelling techniques + data integration patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data More ❯

Position: Analytics Engineer Location: London (hybrid 3 times a week)Department: ITType: 3 months rolling contractOutside IR35 The client is seeking an experienced Analytics Engineer to join its Data & Analytics team. The role focuses on building scalable data pipelines, transforming raw data into clean, reusable datasets, and enabling data-driven decision-making. … Key Responsibilities Design and build data products, with proficiency throughout the data lifecycle. Develop robust data models through close collaboration with business users and the engineering team. Partner with senior management, operational leads, and other stakeholders, coaching and supporting a data-driven culture, including KPI definition and reporting frameworks. Accountable for data extraction, transforming JSON & XML, utlising high experience within metadata management. Collaborate with data engineers to develop data, enrich product design and integrate data for predictive models or machine learning. Deliver well-defined, transformed, tested, documented, and code-reviewed datasets for analysis. Evaluate and recommend improvements in data flow, influencing and More ❯

Engineer Location: London (hybrid 3 times a week) Department: IT Type: 3 months rolling contract Outside IR35 The client is seeking an experienced Analytics Engineer to join its Data & Analytics team. The role focuses on building scalable data pipelines, transforming raw data into clean, reusable datasets, and enabling data-driven decision-making. … Key Responsibilities Design and build data products, with proficiency throughout the data lifecycle. Develop robust data models through close collaboration with business users and the engineering team. Partner with senior management, operational leads, and other stakeholders, coaching and supporting a data-driven culture, including KPI definition and reporting frameworks. Accountable for data extraction, transforming JSON & XML, utlising high experience within metadata management. Collaborate with data engineers to develop data, enrich product design and integrate data for predictive models or machine learning. Deliver well-defined, transformed, tested, documented, and code-reviewed datasets for analysis. Evaluate and recommend improvements in data flow, influencing and More ❯

The Platform & Engineering team at Royal Canin is responsible for managing our data capabilities. This includes the creation, operation, and optimisation of the data platform, assets, and pipelines. This is a growing team, supporting an advanced analytics agenda at Royal Canin that is rapidly transforming it into a business powered by data. To accelerate achieving this … objective, we are looking for enthusiastic data engineers to join our team. The role is responsible for creating and managing trusted Supply data assets. You will be seen as the primary engineering contact for this area, with an expectation to provide expert advice and technical leadership for product squads utilising data assets under your … models to support efficient and accurate analytical insight creation. Reduce data preparation efforts for solution users to expedite their processes and reduce errors. Perform data pipeline migrations if necessary. Implement alerting and monitoring capabilities to ensure high platform reliability in compliance with Mars Cyber Security Standards and Privacy Policies Ensure that product architecture and security More ❯

The Platform & Engineering team at Royal Canin is responsible for managing our data capabilities. This includes the creation, operation, and optimisation of the data platform, assets, and pipelines. This is a growing team, supporting an advanced analytics agenda at Royal Canin that is rapidly transforming it into a business powered by data. To accelerate achieving this … objective, we are looking for enthusiastic data engineers to join our team. The role is responsible for creating and managing trusted Supply data assets. You will be seen as the primary engineering contact for this area, with an expectation to provide expert advice and technical leadership for product squads utilising data assets under your … models to support efficient and accurate analytical insight creation. Reduce data preparation efforts for solution users to expedite their processes and reduce errors. Perform data pipeline migrations if necessary. Implement alerting and monitoring capabilities to ensure high platform reliability in compliance with Mars Cyber Security Standards and Privacy Policies Ensure that product architecture and security More ❯

DATA ARCHITECT - DV CLEARED NEW PERMANENT JOB OPPORTUNITY AVAILABLE WITHIN A LEADING NATIONAL SECURITY SME FOR A DATA ARCHITECT WITH ENHANCED DV CLEARANCE Permanent job opportunity for a Data Architect Leading National Security SME Salary up to £100,000 + Bonus London based organisation in an easily accessible location To apply please call or … email WHO WE ARE? We are recruiting multiple a Data Architect to support urgent National Security & Defence projects in London. Due to the nature of these projects you must hold enhanced DV Security Clearance. WHAT WILL THE DATA ARCHITECT BE DOING? You will be joining a leading SME who is working hard to support National Security … projects within UK Govt. Departments in London. As part of a team, you will be responsible for designing and implementing Data Solutions in Mission-Critical areas. WE NEED THE DATA ARCHITECT TO HAVE.... Current DV clearance - Enhanced Good at understanding complexity and abstracting that into a form that is consumable for a non-technical audience. Experience More ❯

Billigence - Data Engineer Consultant About Billigence: Billigence is a boutique data consultancy with global outreach & clientele, transforming the way organizations work with data. We leverage proven, cutting-edge technologies to design, tailor, and implement advanced Business Intelligence solutions with high added value across a wide range of applications from process digitization through to Cloud Data Warehousing, Visualisation, Data Science, and Engineering or Data Governance. Headquartered in Sydney, Australia with offices around the world, we help clients navigate difficult business conditions, remove inefficiencies, and enable scalable adoption of analytics culture. About the Role: We are seeking a Consultant-Data Engineer to join our growing Practice. You will work … Understanding of data modelling principles, dimensional modelling, and database design Proficiency in SQL and query optimization Comprehensive knowledge of ETL/ELT processes and data pipeline architecture Excellent communication skills with the ability to collaborate across cross-functional teams Experience managing client relationships at various levels Strong problem-solving abilities and analytical thinking Highly Desirable More ❯

Billigence - Data Engineer Consultant About Billigence: Billigence is a boutique data consultancy with global outreach & clientele, transforming the way organizations work with data. We leverage proven, cutting-edge technologies to design, tailor, and implement advanced Business Intelligence solutions with high added value across a wide range of applications from process digitization through to Cloud Data Warehousing, Visualisation, Data Science, and Engineering or Data Governance. Headquartered in Sydney, Australia with offices around the world, we help clients navigate difficult business conditions, remove inefficiencies, and enable scalable adoption of analytics culture. About the Role: We are seeking a Consultant-Data Engineer to join our growing Practice. You will work … Understanding of data modelling principles, dimensional modelling, and database design Proficiency in SQL and query optimization Comprehensive knowledge of ETL/ELT processes and data pipeline architecture Excellent communication skills with the ability to collaborate across cross-functional teams Experience managing client relationships at various levels Strong problem-solving abilities and analytical thinking Highly Desirable More ❯

If you are a natural collaborator and a creative problem solver - this position might be for you! We are looking for a Senior Data Engineer to join our Data team and be a part of Nest's transformation to use its data and insight to drive decision making.You’ll work closely with stakeholders to … develop, test, and maintain data products and solutions that align with their requirements. You’ll be helping to develop and maintain a cloud-based data and analytics platform on Azure and help end-users of every level integrate their workflows between this and other platforms, including AWS-based solutions.You’ll combine data from different … sources, and build data pipelines to transform and organise raw data into formats and structures that can be easily used for business intelligence, visualisations and dashboards, deep dive analysis, and long term forecasting/modelling. To be successful in this positions you will have the following skills, competencies and experience: You’re a natural collaborator and More ❯

Data Engineer (Snowflake) – AI/Data Consulting Firm About Us We are an ambitious consulting firm focused on delivering cutting-edge solutions in data and AI. Our mission is to empower organisations to unlock the full potential of their data by leveraging platforms like Snowflake alongside other emerging technologies. As a Data Engineer, you will play a crucial role in building and optimising data solutions, ensuring scalability, performance, and reliability for our clients’ complex data challenges. The Role As a Data Engineer (Snowflake) , you will be responsible for designing, implementing, and optimising large-scale data processing systems. You’ll work closely with … data accessibility and usability. Optimise data warehouse architectures for performance, scalability, and cost efficiency. Maintain and improve CI/CD processes for data pipeline deployment and monitoring. What We Are Looking For: 5+ years of experience in data engineering or related roles. Strong expertise in Snowflake , SQL , and cloud data More ❯

Data Engineer (Snowflake) – AI/Data Consulting Firm About Us We are an ambitious consulting firm focused on delivering cutting-edge solutions in data and AI. Our mission is to empower organisations to unlock the full potential of their data by leveraging platforms like Snowflake alongside other emerging technologies. As a Data Engineer, you will play a crucial role in building and optimising data solutions, ensuring scalability, performance, and reliability for our clients’ complex data challenges. The Role As a Data Engineer (Snowflake) , you will be responsible for designing, implementing, and optimising large-scale data processing systems. You’ll work closely with … data accessibility and usability. Optimise data warehouse architectures for performance, scalability, and cost efficiency. Maintain and improve CI/CD processes for data pipeline deployment and monitoring. What We Are Looking For: 5+ years of experience in data engineering or related roles. Strong expertise in Snowflake , SQL , and cloud data More ❯

Data Engineer (Databricks) – AI/Data Consulting Firm About Us We are an ambitious consulting firm focused on delivering cutting-edge solutions in data and AI. Our mission is to empower organisations to unlock the full potential of their data by leveraging platforms like Databricks alongside other emerging technologies. As a Data Engineer, you will play a crucial role in building and optimising data solutions, ensuring scalability, performance, and reliability for our clients' complex data challenges. The Role As a Data Engineer (Databricks), you will be responsible for designing, implementing, and optimising large-scale data processing systems. You will work closely with … accessibility and usability. Optimise data warehouse and lakehouse architectures for performance and cost efficiency. Maintain and improve CI/CD processes for data pipeline deployment and monitoring. What We Are Looking For: 5+ years of experience in data engineering or related roles. Strong expertise in Databricks, Spark, Delta Lake, and cloud More ❯

Data Engineer (Databricks) – AI/Data Consulting Firm About Us We are an ambitious consulting firm focused on delivering cutting-edge solutions in data and AI. Our mission is to empower organisations to unlock the full potential of their data by leveraging platforms like Databricks alongside other emerging technologies. As a Data Engineer, you will play a crucial role in building and optimising data solutions, ensuring scalability, performance, and reliability for our clients' complex data challenges. The Role As a Data Engineer (Databricks), you will be responsible for designing, implementing, and optimising large-scale data processing systems. You will work closely with … accessibility and usability. Optimise data warehouse and lakehouse architectures for performance and cost efficiency. Maintain and improve CI/CD processes for data pipeline deployment and monitoring. What We Are Looking For: 5+ years of experience in data engineering or related roles. Strong expertise in Databricks, Spark, Delta Lake, and cloud More ❯