1 to 25 of 197 Data Pipeline Jobs in London

Location: London, Potters Bar or Bristol (Hybrid working options available) The Data Engineer is a hands-on technical role responsible for designing, developing, and maintaining data pipelines within the IT department. The pipelines will be realised in a modern lake environment and the engineer will collaborate … guiding and upskilling more junior data engineers and setting data standards and guidelines. What you'll do Data Pipeline Development Design, model, develop and maintain data pipelines to ingest, store, process, and present data. Ensure data quality, accuracy ...

Service Operations team. This role offers an exciting opportunity to learn, contribute to, and grow within the critical areas of data pipeline development, maintenance, and optimisation for our client-facing products and services. Working closely with experienced developers, data analysts, data scientists … instrumental in ensuring the efficient and reliable flow of data, building foundational skills in a supportive environment. Key Responsibilities Data Pipeline Development • Contribute to the design, building, and maintenance of scalable data pipelines • Implement ETL (Extract, Transform, Load) processes to ingest data ...

About The Role Job level: 10 Were looking for a Lead Data Engineer to join our Data Engineering and Analytics practice. In this role, you will: Lead the design, development, management and optimisation of data pipelines to ensure efficient data flows, recognising … wareho using concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability to communicate technical concepts ...

Robert Half Technology are assisting a market leading AI-driven data platform organisation to recruit a Data Platform Engineer paying c£125 - £150k per annum. Remote working (UK based). Role The Data Platform Engineer will be an expert level engineer within Snowflake … build scalable solutions within the Data Platform Perform root cause analysis on often complex errors to help ensure data pipeline availability Help drive technical & architectural decisions on the data platform including decisions on data architecture, data engineering processes, data ...

unique opportunity to remain hands-on in engineering while also leading a focused, high-impact team. You'll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with … leading global Investment Banks' front-office. This is a rare greenfield role that offers the opportunity to solve the ultimate data pipeline challenge faced by all banks, working closely with various businesses and gaining an overview of many different sectors. What We're Looking For 10 + ...

Data Engineer £Up to £65,000 GBP Hybrid WORKING Location: London; Norwich; Watford; Colchester; Chelmsford; Woking; Chatham; Slough, Central London, Greater London - United Kingdom Type: Permanent Must Have: Active SC Clearance Join a world-class organisation delivering mission-critical data solutions for Defence, National Security … architecture. Familiarity with modern data architectures and cloud-based/distributed systems. Proficiency in SQL, Python and data pipeline tools (e.g., Apache Airflow, Spark). Experience working with major cloud platforms (AWS, Azure, GCP) and big data technologies. Awareness of data ...

Data Engineer Manager Hybrid - London with 2/3 days WFH Circ £85,000 - £95,000 + Attractive Bonus & Benefits This newly created Data Engineer Managers position is an excellent opportunity for someone that enjoys being hands on technically as well as managing a small team … Data Engineers. It would suit those with official management experience, or potentially a Lead or Senior Engineer used to leading teams and now looking to take on more managerial responsibility. Our client is a well-established and rapidly growing global business with its headquarters based in London. ...

Data Engineer Manager Hybrid - London with 2/3 days WFH Circ £85,000 - £95,000 + Attractive Bonus & Benefits This newly created Data Engineer Managers position is an excellent opportunity for someone that enjoys being hands on technically as well as managing a small team … Data Engineers. It would suit those with official management experience, or potentially a Lead or Senior Engineer used to leading teams and now looking to take on more managerial responsibility. Our client is a well-established and rapidly growing global business with its headquarters based in London. ...

Data Engineer (Inside IR35) £425 Per day SC Cleared London o support the formation of a new Data Team, there is a critical need to establish robust, scalable, and secure data infrastructure. Currently, data is dispersed across multiple systems with inconsistent formats … performance, cost efficiency, and maintainability using modern data-engineering tools and platforms (e.g., Azure Data Factory, AWS Data Pipeline, Databricks, Apache Spark). Support the integration of data into visualisation platforms and analytical environments (e.g., Power BI, ServiceNow). Ensure adherence ...

undertaking one of the largest transformation programmes in government, modernising the digital systems that support over £7 trillion of property ownership. As a Data Engineer, you will support the development of HMLR's data engineering capability by helping to build and maintain reliable data … and analysing source systems to produce clear, actionable insights. Desirable Skills Knowledge of DevOps processes: (e.g. Terraform) Knowledge of data pipeline testing (e.g. end-to-end testing, data quality testing, monitoring & alerting, unit & contract testing) Knowledge of the data lifecycle (e.g. ...

undertaking one of the largest transformation programmes in government, modernising the digital systems that support over £7 trillion of property ownership. As a Data Engineer, you will support the development of HMLR's data engineering capability by helping to build and maintain reliable data … and analysing source systems to produce clear, actionable insights. Desirable Skills Knowledge of DevOps processes: (e.g. Terraform) Knowledge of data pipeline testing (e.g. end-to-end testing, data quality testing, monitoring & alerting, unit & contract testing) Knowledge of the data lifecycle (e.g. ...

Data Engineer - £350PD - Remote Required Technical Skills Data Pipeline & ETL Design, build, and maintain robust ETL/ELT pipelines for structured and unstructured data Hands-on experience with AWS Glue and AWS Step Functions Implementation of data validation, data quality frameworks, and reconciliation checks Strong error handling, monitoring, and retry strategies in production pipelines Experience with incremental data processing patterns (CDC, watermarking, upserts) AWS Data Services Amazon S3: data lake architectures, partitioning strategies, lifecycle policies DynamoDB: data modeling, secondary ...

Data Engineer - £350PD - Remote Required Technical Skills Data Pipeline & ETL Design, build, and maintain robust ETL/ELT pipelines for structured and unstructured data Hands-on experience with AWS Glue and AWS Step Functions Implementation of data validation, data quality frameworks, and reconciliation checks Strong error handling, monitoring, and retry strategies in production pipelines Experience with incremental data processing patterns (CDC, watermarking, upserts) AWS Data Services Amazon S3: data lake architectures, partitioning strategies, lifecycle policies DynamoDB: data modeling, secondary ...

Lead Data Engineer Lead Data Engineer required to join my Client's Technology Team and contribute to their Group Architecture Board in building the next generation of their eCommerce data platform.You will play a pivotal role in designing and maintaining a modern, cloud-based … develop robust data models, ETL/ELT processes, and data marts that support complex analytics and operational use cases. Pipeline Optimization: Maintain and optimize data pipelines and warehouse environments to ensure top-tier performance, reliability, and scalability. Cross-Functional Collaboration: Effectively collaborate with ...

Lead Data Engineer Lead Data Engineer required to join my Clients Technology Team and contribute to their Group Architecture Board in building the next generation of their eCommerce data platform. You will play a pivotal role in designing and maintaining a modern, cloud-based … develop robust data models, ETL/ELT processes, and data marts that support complex analytics and operational use cases. Pipeline Optimization: Maintain and optimize data pipelines and warehouse environments to ensure top-tier performance, reliability, and scalability. Cross-Functional Collaboration: Effectively collaborate with ...

Lead Data Engineer Lead Data Engineer required to join my Client's Technology Team and contribute to their Group Architecture Board in building the next generation of their eCommerce data platform. You will play a pivotal role in designing and maintaining a modern, cloud … develop robust data models, ETL/ELT processes, and data marts that support complex analytics and operational use cases. Pipeline Optimization: Maintain and optimize data pipelines and warehouse environments to ensure top-tier performance, reliability, and scalability. Cross-Functional Collaboration: Effectively collaborate with ...

Lead Data Engineer Lead Data Engineer required to join my Client’s Technology Team and contribute to their Group Architecture Board in building the next generation of their eCommerce data platform. You will play a pivotal role in designing and maintaining a modern, cloud … develop robust data models, ETL/ELT processes, and data marts that support complex analytics and operational use cases. Pipeline Optimization: Maintain and optimize data pipelines and warehouse environments to ensure top-tier performance, reliability, and scalability. Cross-Functional Collaboration: Effectively collaborate with ...

Adword Job Title: Data Architect (Insurance Domain) Location: London UK Hybrid role Responsibilities: Experience Level 15 years with at least 5 years in Azure ecosystem Role: We are seeking a seasoned Data Architect to lead the design and implementation of scalable data solutions … strategic insurance project The ideal candidate will have deep expertise in Azure cloud services Azure Data Factory and Databricks with a strong understanding of data modeling data integration and analytics in the insurance domain Key Responsibilities Architect and design end to end data ...

Adword Job Title: Data Architect (Insurance Domain) Location: London UK Hybrid role Responsibilities: Experience Level 15 years with at least 5 years in Azure ecosystem Role: We are seeking a seasoned Data Architect to lead the design and implementation of scalable data solutions … strategic insurance project The ideal candidate will have deep expertise in Azure cloud services Azure Data Factory and Databricks with a strong understanding of data modeling data integration and analytics in the insurance domain Key Responsibilities Architect and design end to end data ...

advisers and their clients by combining our innovative proprietary technology with exceptional customer service. As we scale, we are growing our multi-disciplinary Data, Analytics&AI function to unlock the full potential of our data assets. About the Role This is a high-impact hands … leadership role with the opportunity to design and build Fundment's data and analytics infrastructure from the ground up. Reporting to our Head of Data&Analytics, you will create the data foundations that allow us to deliver data-driven insights, automated systems ...

Data Engineer £250 Outside IR35 - Remote Job Summary We are seeking a skilled Azure Data Engineer to design, build, and maintain scalable data solutions on Microsoft Azure. The ideal candidate has strong hands-on experience with Azure Databricks and Azure Synapse Analytics … stakeholders to deliver data solutions Implement security, governance, and compliance best practices (RBAC, data masking, encryption) Monitor, troubleshoot, and resolve pipeline and performance issues Document data architecture, pipelines, and operational processes Required Qualifications 3+ years of experience as a Data Engineer ...

Data Engineer £250 Outside IR35 - Remote Job Summary We are seeking a skilled Azure Data Engineer to design, build, and maintain scalable data solutions on Microsoft Azure. The ideal candidate has strong hands-on experience with Azure Databricks and Azure Synapse Analytics … stakeholders to deliver data solutions Implement security, governance, and compliance best practices (RBAC, data masking, encryption) Monitor, troubleshoot, and resolve pipeline and performance issues Document data architecture, pipelines, and operational processes Required Qualifications 3+ years of experience as a Data Engineer ...

Role Title: Data Engineer Role Type: Contract (inside ir35) Role Location: London, UK Mandatory: Active Security Clearance Job Description:- Essential Skills & Experience: ·10+ years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks. ·Good proficiency in Python … Database. ·Experience working with large datasets and complex data pipelines. ·Experience with data architecture design and data pipeline optimization. ·Proven expertise with Databricks, including hands-on implementation experience and certifications. ·Experience with SQL and NoSQL databases. ·Experience with data quality ...

pipelines and transformation workflows using Databricks and Python. Ensure data platforms support analytics, business intelligence, and emerging GenAI/ML needs. Pipeline Development & Workflow Management Develop, optimise, and monitor ETL/ELT pipelines across various data sources. Implement data validation, governance, quality … engineering within the organisation. Ensure data models and pipelines effectively support reporting and decision-making. Performance & Continuous Improvement Monitor system and pipeline performance, ensuring SLAs are met. Evaluate and implement new technologies and tools to improve data capabilities. Automate and streamline processes to enhance ...

Essential Skills & Experience: 10+ years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks . Good proficiency in Python and Spark (PySpark) or Scala. Deep understanding of data warehousing principles, data modelling techniques, and data … Database . Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks , including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality ...