76 to 100 of 1,537 Data Pipeline Jobs in London

challenges, driving innovation, and ensuring the seamless delivery of data solutions. What you'll be doing: Primary Responsibilities: Client Engagement & DeliveryData Pipeline Development (Batch and Streaming)Snowflake & Cloud Data PlatformsData Architecture & ModellingCollaboration & Best PracticesQuality, Governance & Security Business Relationships: Solution ArchitectsData Engineers, Developers, ML Engineers … Data Architect, and Analytics leadership What experience you'll bring: Competencies/Critical Skills: Proven experience in data engineering and pipeline development on Snowflake and cloud-native platforms. Strong consulting values with ability to collaborate effectively in client-facing environments. Hands-on expertise across ...

Lead Data Engineer or architect at Made Tech, you'll play a pivotal role in helping public sector organisations become truly data‐lead, by equipping them with robust data platforms. You'll also join a data team on its mission … data knowledge and skills out of silos and embedded into delivery teams. You'll also help implement efficient data pipelines & storage. Key responsibilities Define, shape and perfect data strategies in central and local government. Help public sector teams understand the value of their data ...

Fabric Consultant Up to £90,000 We are partnering with a leading Microsoft-focused consultancy that specialises in delivering enterprise-scale data, analytics and AI transformation projects. Working with organisations across a range of industries, the business helps clients unlock greater value from their data through … quality solutions while contributing to innovation and best practice across multiple client projects. Key Responsibilities Design, build and optimise high-performance data pipeline solutions using Microsoft Fabric, Azure Synapse and Azure Databricks. Develop scalable data platforms to ingest, store and transform large volumes of data. ...

transformation within the AI landscape, ensuring the adoption of cutting-edge technologies and compliance with global control and regulatory frameworks. The AI and Data Engineer role is responsible for designing, building, and operating data pipelines, data products and AI-enabled engineering components that prepare … continually improve operational stability and performance. Security, Governance & Controls Embed secure data access mechanisms and fine-grained controls directly into pipelines, data products and AI-facing services. Implement lineage, auditability, provenance, privacy and retention controls consistent with internal standards and regulatory expectations. Support AI governance activities ...

profitable. We’ve done a pretty good job so far, but we’re just getting started! London | 2 days per week in office Data Engineering Team The Data Platform team is responsible for designing, building, maintaining, and optimising our modern data platforms and infrastructure. … quality, reliable, and performant data, from ingestion through to consumption by various teams. We work on key projects involving data pipeline development, platform management, model training and deployment, data quality assurance and customised data tooling. What You’ll Be Doing Design ...

relaxed atmosphere, regular social events and remarkable colleagues looking to push technology boundaries - come and join us! The Role As a Quant Data Engineer, you will be joining a small but growing team working closely with Pythia’s quant modellers to ensure they have access to reliable, well … structured and high-quality data for research, modelling and analysis. With particular emphasis on data quality, usability, investigation and continuous improvement, you will work side-by-side with quant, engineering and operational colleagues to prepare datasets, improve underlying data flows and help ensure that ...

Financial Crime Data Lead Hybrid: Central London 2 days a week in the office Permanent Paying up to £120k + Bonus We are currently working with a leading organisation undergoing a major transformation of its Financial Crime, Risk and Compliance technology landscape. As part of this journey, they … looking to hire a Financial Crime Data Lead to play a central role in designing and building a new enterprise-wide Financial Crime data platform that will underpin key capabilities including transaction monitoring, client screening, onboarding, risk assessment and future AI-driven decision making. The Role ...

considerable responsibility and ownership of your path. At the core, it involves: Data engineering and analysis – extending the data pipeline which feeds our carbon market dashboard and bespoke analytics platform Developing the back-end code and infrastructure for the API – providing access to our offsets … alternative finance to emerging markets Responsibilities You will be working on the following: Extending and maintaining the codebase for the data pipeline which feeds into our carbon offsets database Data gathering, processing and analysis; researching and implementing ways to unlock data from ...

Azure Data Stack, critically ADF and Synapse (experience with Microsoft Fabric is a plus) Highly developed python and data pipeline development knowledge, must include substantial PySpark experience Demonstrable DevOps and DataOps experience with an understanding of best practices for engineering, test and ongoing service delivery … understanding of Infrastructure as Code concepts (Demonstrable Terraform experience a plus) Demonstrable experience of Data Pipeline testing, including automated testing, data validation and code assurance Demonstrable experience of working within Agile Delivery projects An understanding of data formats for ingest, transformation and analytics ...

About Rayo Rayo, part of UBDS Group, helps organisations solve complex technology challenges across cloud, data, AI and digital transformation. We work with clients operating in complex, secure and highly regulated environments, with a significant focus on the public sector and defence. We’re looking for an experienced … Senior Data Engineer with strong AWS expertise to design, build and deliver scalable, secure and resilient data solutions for our clients. This is a hands‐on senior engineering role, combining deep data engineering expertise with technical ownership, client engagement and the ability to guide ...

analysts, AI engineers, and platform leads to deliver production‐grade, business‐critical data products. Lead by example in data documentation, pipeline testing, and lineage tracking — ensuring transparency and reproducibility across all pipelines. Qualifications Hands‐on experience as a Data Engineer or Data Platform Developer, including direct contributions to production codebases. Expert‐level proficiency in Python, SQL, and Apache Spark for data pipeline development. Proven experience building ELT workflows using Fivetran, Airflow, DBT, or Databricks. Solid understanding of data modeling, dimensional design, and schema normalization ...

Financial Crime Data Lead Hybrid: Central London 2 days a week in the office Permanent Salary £100,000 - £120,000 pa Paying up to £120k + Bonus Posted 1 Sep 2026 (Last week) We are currently working with a leading organisation undergoing a major transformation of its Financial … Crime, Risk and Compliance technology landscape. As part of this journey, they are looking to hire a Financial Crime Data Lead to play a central role in designing and building a new enterprise-wide Financial Crime data platform that will underpin key capabilities including transaction monitoring ...

flow across the organisation. Build efficient, fault-tolerant, and highly available data ingestion processes. Monitor and improve data pipeline performance and scalability for low-latency and high-throughput. Work closely with cross-functional teams, including software engineers, data scientists, and business … similar language. Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling. Strong understanding of data pipeline orchestration tools (e.g., Apache Airflow, Kubernetes). You thrive when working as part of a team Comfortable in a fast-paced environment Have excellent written ...

recruiting on behalf of our client for an experienced Senior Data Engineer with strong AWS expertise to design, build and deliver scalable, secure and resilient data solutions. Our client works with organisations operating in complex, secure and highly regulated environments, with a significant focus … public sector and defence. This is a hands‐on senior engineering role, combining deep data engineering expertise with technical ownership, client engagement and the ability to guide other engineers and influence engineering practices. The Role As a Senior Data Engineer, you’ll take a leading role ...

UNITED is your playing field. Join us in shaping the future of gaming. Are you ready to LEVEL-UP THE GAME? Key Responsibilities Data Platform & ArchitectureDevelop and maintain robust and efficient data solutions and products including data transformation, data modelling, and reporting … Experience with ETL/ELT processes and data ingestion Proven expertise using transformation tools such as dbt Experience with data pipeline orchestration tools (such as Apache Airflow) Experience with best practices applied to data (e.g., testing, CI/CD) Advantageous Experience developing dashboards ...

e.g. SSIS or equivalent tools and frameworks). Programming (e.g. Python) – Experience using Python or similar languages for data processing, automation, or pipeline development. Cloud & Platform Technologies – Familiarity with modern data platforms and cloud-based architectures (desirable). Performance & Scalability Optimisation – Ability to design … modelling, system design, and integration patterns. Strong experience with SQL and data processing at scale. Experience working with data pipeline and ETL tools (e.g. SSIS or equivalent). Experience with programming languages (e.g. Python) for data processing or automation. Experience defining ...

Data, AI, Solutions & Engineering (DAISE) practice is looking for an experienced Data Engineer to join the team. In DAISE, we are focused on delivering value‐adding, sustainable data capabilities, aligned to our client’s specific needs. This expertise is applied across clients … Understanding of key core concepts like distributed computing, batch & stream processing, functional and object‐orientated programming, how pipelines are built and deployed on cloud, pipeline schedules and SLAs Well‐versed with documentation and artefacts that need to go along with the solution design and delivery work #J-18808-Ljbffr ...

Title Data Governance Lead Job Description About IG Group IG is a FTSE 100 fintech operating across five continents, serving over 1.3m customers and handling billions of dollars in transactions – built on scale, trust, and proof. We didn't pivot to innovation; it's how we've always … give you the platform to define what comes next. Join us at IG – the future gets built here. About the Role IG's Data & AI team is scaling up how the business governs, understands, and trusts its data. As Data Governance Lead, you'll turn ...

Digital Innovation, R&D London, United Kingdom Your new role As a Sr Data Engineering Professional, you will play a key role in developing and advancing data pipelines & data products that support early research activities. You will work at the intersection of engineering excellence … embedding AI, data science, and data at the heart of drug discovery and development. This includes accelerating the pipeline through computational methods and digital innovation. By leveraging data and AI, we aim to quickly identify drug candidates, reduce development time and costs ...

DATA strives to hire exceptional, innovative and passionate individuals who want to grow with us. If you want to be part of an inclusive, adaptable, and forward-thinking organization, apply now. Role SummaryWe are seeking a highly skilled Senior Data Engineer to design, build, and maintain … Azure Data Factory/Synapse/DatabricksAWS Glue/Redshift/EMRGoogle BigQuery/DataflowEnsure systems are secure, scalable, and cost-efficientData Pipeline & Workflow ManagementDevelop orchestration workflows (e.g., Airflow, Prefect, Azure Data Factory)Monitor and troubleshoot pipelines to ensure high availability and performanceImplement robust error ...

pretty good job so far, but we're just getting started! ð London | ð 2 days per week in officeData Engineering Team ðThe Data Platform team is responsible for designing, building, maintaining, and optimising our modern data platforms and infrastructure. Our main goal is to ensure … quality, reliable, and performant data, from ingestion through to consumption by various teams. We work on key projects involving data pipeline development, platform management, model training and deployment, data quality assurance and customised data tooling. What You'll Be Doing ðDesign ...

Retrieval-Augmented Generation) architectures that ground LLM outputs in Comply'sproprietary dataEvaluate and integrate LLM tooling and frameworks appropriate to Comply's use casesData Pipeline & Platform EngineeringBuild reliable, observable data pipelines that feed the semantic layer from upstream broker andregulatory data sourcesApply DataOps practices including … integrated data pipelinesFamiliarity with semantic web standards — JSON-LD, RDF, OWL, or SKOSStrong Python skills and experience with data pipeline frameworksExperience with cloud-native data platforms (AWS, Azure, or GCP)Exposure to domain-driven design (DDD) and bounded contexts is desirable. Experience working ...

development of applications that integrate and leverage AI/ML models, from architectural design, data schema design, data pipeline construction, and rapid prototyping to initial deployment and operationalization, utilizing cloud-native services (e.g., serverless, containerization, managed AI/ML platforms) and CI/CD pipelines … practices to streamline model deployment, monitoring, and lifecycle management in cloud environments, including data versioning, feature store integration, and data pipeline management. Business Partnership & Solution Architecture: Collaborate closely with business and engineering teams to deeply understand their challenges and customer needs, identify high-impact opportunities ...

status, or any basis prohibited by the laws that govern its operations. How does this role contribute to our collective success? As a Data Quality Business Partner within AXIS, you will support the Data & Analytics Transformation programme, which aims to replace our legacy data estate and modernise it through a new data platform. Your role will focus on helping ensure the integrity, accuracy, and usability of data across the organisation. Working closely with technical teams and business stakeholders, you will apply data quality best practices ...

where we’re headed. We’re proud to share our story and Make Amazing Happen at CDW. JOB TITLE: Senior Data Engineer DEPARTMENT: Enterprise Data LOCATION: One New Change, London, EC4M 9AF SHIFT PATTERN: Monday to Friday, 09:00-17:30 - Hybrid working (3 days … Office per week) RESPONSIBLE FOR: Leading (and carrying out) Data Engineering activities for and not limited to the Data Platforms and Services BACKGROUND: CDW is a leading technology solutions provider to business, government, education and healthcare organisations in the United Kingdom, Canada, and the United States. ...