76 to 100 of 987 Data Pipeline Jobs in England

Senior Databricks Data Engineer 2-3 days on-site - central London up to £75k + 20% Bonus + Excellent Benefits Our client is a leading global Retail/FMCG brand undergoing an exciting period of rapid growth and transformation. With significant investment in data and technology, they are building a world-class data platform … power decision-making across every area of the business - from supply chain and logistics to marketing, customer sales and in-store operations. We are seeking an experienced Senior Data Engineer with deep expertise in Databricks and Medallion Architecture (Bronze-Silver-Gold) to design, build, and optimize our next-generation data platform. This role will be pivotal … in developing scalable data pipelines, enabling advanced analytics, and driving data quality and governance across the organization. You'll work closely with data scientists, analysts, and business stakeholders to transform raw data into trusted, actionable insights that power critical business decisions. Key Responsibilities Design and implement scalable data pipelines and More ❯

Looking for a Data Engineer to join the Data Engineering team. The team is responsible for ensuring the delivery of accurate, curated data for downstream consumption in a timely manner, and are responsible for extracting data from source solutions and transforming that into a central data repository The role will … be working closely with the Data Analytics team, providing assistance where required on producing any required outputs needed by the business. Perform the day to day running of the ETL processes that feed into the central data repository. Work with key stakeholders and other teams to gather requirements, identify where the data is located … them in a data and reporting solution. The ability to analyse, refactor and implement processes (technical/business). Good technical understanding of developing data pipeline solutions abiding to best practice. Good communication skills. Ability to document and accurately capture business requirements, translating that into a Technical Solution Specification document detailing the data More ❯

Looking for a Data Engineer to join the Data Engineering team. The team is responsible for ensuring the delivery of accurate, curated data for downstream consumption in a timely manner, and are responsible for extracting data from source solutions and transforming that into a central data repository The role will … be working closely with the Data Analytics team, providing assistance where required on producing any required outputs needed by the business. Perform the day to day running of the ETL processes that feed into the central data repository. Work with key stakeholders and other teams to gather requirements, identify where the data is located … them in a data and reporting solution. The ability to analyse, refactor and implement processes (technical/business). Good technical understanding of developing data pipeline solutions abiding to best practice. Good communication skills. Ability to document and accurately capture business requirements, translating that into a Technical Solution Specification document detailing the data More ❯

Data Engineer – Python | SQL | Azure | Microsoft Fabric Location: London Salary: £90,000 + benefits Financial Services Are you a skilled Data Engineer with strong coding expertise and a passion for building scalable data solutions? We’re looking for a talented professional to join our team and play a key role in designing, developing, and … optimising modern data platforms on Azure Cloud — with a focus on an exciting Microsoft Fabric implementation. What you’ll do Design and build robust data pipelines and ETL processes using Python and SQL. Contribute to the rollout and optimisation of Microsoft Fabric, shaping the next-generation data platform. Work across Azure services (Data Factory, Synapse, Databricks, Data Lake, Functions) to deliver secure and scalable data solutions. Optimise data workflows for performance, reliability, and cost efficiency. Collaborate with analysts, data scientists, and business stakeholders to ensure high-quality, trusted data is available for decision-making. Implement best practices for data More ❯

Data Engineer – Python | SQL | Azure | Microsoft Fabric Location: London Salary: £90,000 + benefits Financial Services Are you a skilled Data Engineer with strong coding expertise and a passion for building scalable data solutions? We’re looking for a talented professional to join our team and play a key role in designing, developing, and … optimising modern data platforms on Azure Cloud — with a focus on an exciting Microsoft Fabric implementation. What you’ll do Design and build robust data pipelines and ETL processes using Python and SQL. Contribute to the rollout and optimisation of Microsoft Fabric, shaping the next-generation data platform. Work across Azure services (Data Factory, Synapse, Databricks, Data Lake, Functions) to deliver secure and scalable data solutions. Optimise data workflows for performance, reliability, and cost efficiency. Collaborate with analysts, data scientists, and business stakeholders to ensure high-quality, trusted data is available for decision-making. Implement best practices for data More ❯

Join us as a Data Platform Engineer for our client. At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector – we are always in search of the best … you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries. The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and on … with over 3 years of relevant experience in data engineering, platform engineering, or a related field, with demonstrated hands-on expertise in NiFi and data pipeline design in regulated environments. Responsibilities: Design, develop, and maintain robust and secure data pipelines using NiFi and related big data technologies. Troubleshoot and optimize NiFi More ❯

We are looking for a Lead Data Architect to provide strategic leadership and technical expertise in the design and development of data solutions. This role involves setting technical standards, evaluating new technologies, and shaping our approach to data. You will lead a team of Data Architects, Engineers, and Analysts, defining data architecture … frameworks and ensuring the development of scalable, robust, and cost-effective solutions. The ideal candidate will have a passion for data, be technology-agnostic, and possess excellent client-facing skills. Lead Data Architect Annual Salary: circa £85,000 Location: Flexible, hybrid working with some client travel Day to day of the role: Shape the approach to … platforms in Azure/AWS or significant on-premise design experience. 5+ years in data engineering or business intelligence roles. Extensive ETL and data pipeline design experience, technology agnostic. Proficiency in SQL and experience with data engineering coding languages such as Python, R, or Spark. Understanding of data warehouse and More ❯

you, this means working on challenging projects that demand innovation, collaboration, and thought leadership . Role – We are looking for a talented Analytics Engineer to join our growing data and analytics team. You will play a pivotal role in shaping the organisation’s data-driven strategy, working across data engineering, analytics, and business domains … to design scalable, high-quality data models and pipelines. This is a fantastic opportunity to work on cutting-edge projects using technologies such as Snowflake, dbt, Data Vault, and Azure/AWS , while collaborating with data architects, data scientists, and business stakeholders to deliver insights that drive measurable impact. Key Responsibilities: Act … as the bridge between engineering teams and business stakeholders , translating requirements into data solutions. Design, build, and maintain data pipelines and transformations that turn raw data into structured, analysis-ready datasets. Develop and optimise data models (including Data Vault and dimensional modelling) to support large-scale analytics. Implement and enforce More ❯

Robert Half have partnered with a market leading property development organisation for an experienced Azure Data Engineer to join their data and analytics team, on an initial 3-6 month contract based out of their London office The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines in … the Azure cloud environment, with hands-on experience in modern data platforms and ETL frameworks. Key Responsibilities: Design, develop, and maintain data pipelines and data integration solutions using Azure Data Factory, Synapse Analytics, and related services. Build and optimise data models to support reporting, analytics, and machine learning use cases. … Implement robust data quality, governance, and security practices across the data ecosystem. Collaborate with analysts, data scientists, and business stakeholders to understand data needs and translate them into technical solutions. Work with structured and unstructured data sources, ensuring scalability, reliability, and performance. Monitor and troubleshoot data workflows, ensuring More ❯

leader who specialises in buy-side brokerage services - their software provides portfolio managers with differentiated trade idea creation, optimisation, and execution services. You will: Develop and maintain robust data pipelines. Integrate and manage market data feeds. Optimise data storage and retrieval processes for efficiency. Enhance and create advanced data visualisation applications. … Requirements: Proficient in Software Python Development. 3-5 years experience in software engineering Experience with libraries/frameworks such as Pandas, Numpy, Scipy, etc. Skilled in data pipeline orchestration management libraries (e.g., Airflow, Prefect). Experience with cloud infrastructure (AWS, GCP, Azure). DevOps skills (CI/CD, containerisation). Familiarity with Plotly Dash for interactive data visualisations. You will play a pivotal role in supporting and scaling their data platform, ensuring seamless data pipeline orchestration, storage and compute optimization, as well as providing high performance data access interface to support analytical software. This is a fantastic opportunity to lead development of the critical part of product architecture. More ❯

leader who specialises in buy-side brokerage services - their software provides portfolio managers with differentiated trade idea creation, optimisation, and execution services. You will: Develop and maintain robust data pipelines. Integrate and manage market data feeds. Optimise data storage and retrieval processes for efficiency. Enhance and create advanced data visualisation applications. … Requirements: Proficient in Software Python Development. 3-5 years experience in software engineering Experience with libraries/frameworks such as Pandas, Numpy, Scipy, etc. Skilled in data pipeline orchestration management libraries (e.g., Airflow, Prefect). Experience with cloud infrastructure (AWS, GCP, Azure). DevOps skills (CI/CD, containerisation). Familiarity with Plotly Dash for interactive data visualisations. You will play a pivotal role in supporting and scaling their data platform, ensuring seamless data pipeline orchestration, storage and compute optimization, as well as providing high performance data access interface to support analytical software. This is a fantastic opportunity to lead development of the critical part of product architecture. More ❯

Who We Are BCG Expand is a specialist firm providing unique data-driven business intelligence as a trusted advisor to senior executives across the world's leading financial services firms to help them to operate more effectively. Expand's activities are focused on data benchmarking in technology, operations, FinTech and market data, with expertise … our professional, supportive and entrepreneurial working culture that we all enjoy. What You'll Do We're looking for a highly skilled Principal Engineer with a passion for data, cloud technologies, and building robust, scalable solutions - who also brings strategic product thinking and cross-functional leadership . In this role, you'll not only lead the design and … implementation of the data infrastructure that powers advanced analytics and data science across our platforms, but also contribute to the strategic direction, roadmap, and execution of data products in collaboration with senior stakeholders. Architect & Build Scalable Data Solutions Collaborate closely with senior product stakeholders to understand data needs and architect More ❯

Senior Data Engineer Hybrid 2x per week (Leeds, West Yorkshire) Salary to £65,000 + benefits About the Role: Our client, a tech for good organisation, is investing heavily in digital transformation and building a cloud-first, data-led approach to support major environmental goals. Think scalable pipelines, secure platforms, and data systems that … directly contribute to a greener future. As a Senior Data Engineer, you'll play a key role in designing and delivering robust, cloud-based data infrastructure. You'll work with large datasets, build scalable pipelines, and help ensure data is high quality, secure, and accessible, while mentoring more junior engineers and working closely with … stakeholders across the business. Key Responsibilities: Develop scalable and secure data architectures ensuring optimal data flow Cloud data platform development, data modelling, shaping and technical planning Implementing best practices around data security, governance, privacy, and compliance Ensure company data is secure, accurate, maintained and available. Mentor data More ❯

Active SC Candidate only Experience 10 Years with relevant 6-8+ Years Essential Skills & Experience: 10+ years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks . Good proficiency in Python and Spark (PySpark) or Scala. Deep understanding of data warehousing principles, data modelling techniques … and data integration patterns. Extensive experience with Azure data services, including Azure Data Factory, Azure … Blob Storage, and Azure SQL Database . Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks ,including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

TL;DR Kharon is seeking a full-time Data Engineer based in London. This role requires in-office attendance at least 4 days per week, with additional flexibility awarded after complete onboarding. RESPONSIBILITIES: Help build large-scale distributed automated data processing systems and data lakes, while optimizing for both computational and storage efficiency on … AWS Databricks. Create data pipelines, infrastructure, and overall workflow orchestration to pull data from a diverse set of data sources into the Kharon Data Lake. Build systems to track, monitor and validate the quality of data coming into or going out of Kharon Data Lake. Collaborate with engineers … Experience in Python, Pandas, PySpark, and Notebooks. SQL knowledge and experience working with relational databases including schema design, access patterns, query performance optimization, etc. Experience with data pipeline technologies like AWS Glue, Airflow, Kafka, or other cloud based equivalence. Experience with ETL and data warehousing like Databricks, Snowflake, or equivalent. Container-based deployment experience using More ❯

Following the acquisition of Brookson by People2.0, the responsibilities of the Data and Analytics team have significantly expanded. We are now seeking a skilled Data Engineer to join our dynamic team. As a Data Engineer within the Data and Analytics team, you will play a pivotal role in designing, maintaining, and optimising … the data architecture for People2.0 Group. This architecture ensures the seamless flow of data from source systems to the Data Warehouse, providing a robust foundation of aggregated analytical base tables and operational sources. Our data architecture is critical in enabling People2.0 Group to become a data-driven organisation. Your responsibilities … will include maintaining existing data pipelines, developing new branches within the architecture, and collaborating closely with stakeholders across the business. You will engage with stakeholders from various regions, including EMEA, US, and APAC, ensuring that the architecture aligns with regional and global requirements. Reporting directly to the Senior Data Engineer within the Data and More ❯

James's Place, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. The Data Engineering and Analytics Apprenticeship Programme Assessment Centre: Tuesday 17 Feb 2026 The Data Engineering and Analytics Apprenticeship Programme within our CTO Function is a 12-month programme … and offers an excellent grounding in the fundamental principles of Data in one of the UK's largest wealth management organisations. The Chief Data Office (CDO) sits within our Chief Technology Office (CTO) and ensures data is managed as one of the organisation's most valuable assets. It sets the vision for how data … information is accurate, well-managed, and secure, and helps the business unlock value through insights and innovation. As an apprentice in this area, you'll be exposed to: Data governance and quality - learning how we make sure data is trusted and consistent. Data management - understanding how data is stored, integrated, and shared More ❯

New Permanent Opportunity for an eDV Cleared Data Engineer with AWS Cloud Experience for a leading National Security Consultancy in London. Up to £85,000 DoE plus bonuses Active eDV required London Location – full-time on-site when required Expertise required in AWS, Data Pipelines, ETL, Data Storage, and DevOps methodologies Role Overview: This … role sits within our client’s rapidly growing Cloud Data Platforms team, part of the Insights and Data Global Practice. You will join a multidisciplinary group of data and platform specialists who deliver modern cloud-based transformation for clients across a range of sectors. In this role, you will design and build data pipelines, develop ETL/ELT processes, and create innovative data solutions using the latest cloud technologies and frameworks across AWS. Some responsibilities include: Build data pipelines to ingest, process and transform data for analytics and reporting. Develop ETL/ELT workflows to move data efficiently into data warehouses More ❯

New Permanent Opportunity for an eDV Cleared Data Engineer with AWS Cloud Experience for a leading National Security Consultancy in London. Up to £85,000 DoE plus bonuses Active eDV required London Location – full-time on-site when required Expertise required in AWS, Data Pipelines, ETL, Data Storage, and DevOps methodologies Role Overview: This … role sits within our client’s rapidly growing Cloud Data Platforms team, part of the Insights and Data Global Practice. You will join a multidisciplinary group of data and platform specialists who deliver modern cloud-based transformation for clients across a range of sectors. In this role, you will design and build data pipelines, develop ETL/ELT processes, and create innovative data solutions using the latest cloud technologies and frameworks across AWS. Some responsibilities include: Build data pipelines to ingest, process and transform data for analytics and reporting. Develop ETL/ELT workflows to move data efficiently into data warehouses More ❯

On behalf of our prestigious financial services client, we are seeking an experienced and strategic Data Architect to lead the development and delivery of enterprise-wide data governance and architecture initiatives. This is a high-impact role that will shape the organisation's approach to data management, compliance, and innovation. Key Responsibilities: Strategic Framework … Governance Development Design and implement a robust data governance framework, including policies, standards, and procedures. Define data ownership models and appoint data stewards across key domains. Introduce and manage a metadata catalogue to enhance data discoverability and transparency. Establish clear retention, access, and usage policies for various data types. Governance … Leadership Set up and lead a Data Governance Council to oversee governance strategy and execution. Collaborate with data owners, stewards, and custodians across departments to ensure consistency and accountability. Data Quality & Compliance Develop and maintain data quality standards and performance metrics. Ensure full compliance with data protection regulations, including GDPR More ❯

Pay: £450 - £550 per day Location - London (Hybrid) | Python | ETL | Impact-Driven Team We are partnering with a leading client seeking an Azure Data Engineer to join their team on an initial 6-month contract . This hybrid role requires 2 days per week onsite , offering the opportunity to design, build, and maintain cutting-edge data platforms.Data Engineering & Architecture: Design, build, and maintain scalable data pipelines and ETL/ELT processes using Azure services. Develop and optimize data lake and data warehouse solutions (eg, Azure Data Lake Storage, Azure Synapse Analytics). Implement best practices in data modelling, partitioning, and performance optimization. Support Real … Time and batch data processing workloads. Data Quality, Governance & Security Implement data validation, monitoring, and quality frameworks to ensure clean and trustworthy data. Work with the data governance team to ensure compliance with standards, policies, and privacy regulations. Maintain metadata, lineage, and documentation for all data solutions. Collaboration & Stakeholder Support More ❯

Are you passionate about Data and Analytics (D&A) and excited about how it can completely transform the way an enterprise works? Do you have the strategic vision, technical expertise, and leadership skills to drive data-driven solutions? Do you want to work in a dynamic, fast-growing category? If so, you might be the ideal … candidate for the role in the Data and Analytics function for Global Pet Nutrition (PN) at Mars. Pet Nutrition (PN) is the most vibrant category in the FMCG sector. As we work to transform this exciting category, a new program, Digital First, has been mobilized by the Mars Pet Nutrition (PN) leadership team. Digital First places pet parents … demonstrate significant thought leadership, quality decision making, deep technical know-how, and an ability to navigate complex business challenges while building and leading a team of world class data and analytics leaders. With Digital First, PN is moving to a Product based model to create business facing digital capabilities. Develop and maintain robust data pipelines and More ❯

Are you passionate about Data and Analytics (D&A) and excited about how it can completely transform the way an enterprise works? Do you have the strategic vision, technical expertise, and leadership skills to drive data-driven solutions? Do you want to work in a dynamic, fast-growing category? If so, you might be the ideal … candidate for the role in the Data and Analytics function for Global Pet Nutrition (PN) at Mars. Pet Nutrition (PN) is the most vibrant category in the FMCG sector. As we work to transform this exciting category, a new program, Digital First, has been mobilized by the Mars Pet Nutrition (PN) leadership team. Digital First places pet parents … demonstrate significant thought leadership, quality decision making, deep technical know-how, and an ability to navigate complex business challenges while building and leading a team of world class data and analytics leaders. With Digital First, PN is moving to a Product based model to create business facing digital capabilities. Develop and maintain robust data pipelines and More ❯

New Permanent Opportunity available with a leading National Security Organisation for an eDV cleared Data Engineer in Gloucester Gloucester location – hybrid working when possible Must hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who … to the National Security sector, delivering mission-critical solutions that help keep the nation safe, secure, and prosperous. You’ll work with cutting-edge technologies including AI/Data Science, Cyber, Cloud, DevOps/SRE, and Platform Engineering. They have long-term contracts secured across the latest customer framework and are set for significant growth. What will the … Data Engineer be Doing? You will develop mission-critical data solutions and manage pipelines that transform diverse data sources into valuable insights for our client’s National Security customers. You will collaborate with clients to solve complex challenges, utilising distributed computing techniques to handle large-scale, real-time, and unstructured data. Responsibilities include: Design More ❯

Senior Data Analyst Location: Hybrid (2/3 day per week in our Heald Green office) Salary: Up to £65,000 depending on experience About My Claim Group My Claim Group is a fast-growing UK-based company helping customers navigate complex financial and insurance claims. Our success is built on trust, innovation, and data-driven … decision-making - and now we're looking to expand our analytics team to support smarter, faster business outcomes. We're on the lookout for a talented Senior Data Analyst with strong SQL skills and exposure to Snowflake. This is a fantastic opportunity to join a growing team, create meaningful insights, and influence decisions across the business. The Role … experience in data visualisation (Power BI ideal). Solid SQL skills and comfort working with large, complex datasets. Strong grasp of ETL concepts and data pipeline processes. Proficiency in data cleaning, transformation, and preparation. Excellent communication and data storytelling ability. Ability to collaborate with stakeholders and translate requirements into analytical solutions. More ❯