801 to 825 of 1,015 Data Pipeline Jobs in London

Position: Data EngineerLocation: Central London or Bristol (3 days per week onsite)Salary: £600 to £650 p.d inside IR35Security Clearance: SC Clearance Required About the Role A leading IT consultancy delivering innovative solutions to the UK public sector is seeking an experienced Data Engineer to support a high-profile programme. You'll collaborate with cross-functional … teams to build scalable data pipelines and contribute to digital transformation initiatives across government departments. Key Responsibilities Design, develop and maintain robust data pipelines using PostgreSQL and Airflow or Apache Spark Collaborate with frontend/backend developers using Node.js or React Implement best practices in data modelling, ETL processes and performance optimisation Contribute to … Docker/Kubernetes) - experience in this area is desirable, not essential Operate within Agile teams and support DevOps practices What We're Looking For Proven experience as a Data Engineer in complex environments Strong proficiency in PostgreSQL and either Airflow or Spark Solid understanding of Node.js or React for integration and tooling Familiarity with containerisation technologies (Docker/ More ❯

Position: Data Engineer Location: Central London or Bristol (3 days per week onsite) Salary: £600 to £650 p.d inside IR35 Security Clearance: SC Clearance Required About the Role A leading IT consultancy delivering innovative solutions to the UK public sector is seeking an experienced Data Engineer to support a high-profile programme. You'll collaborate with … cross-functional teams to build scalable data pipelines and contribute to digital transformation initiatives across government departments. Key Responsibilities Design, develop and maintain robust data pipelines using PostgreSQL and Airflow or Apache Spark Collaborate with frontend/backend developers using Node.js or React Implement best practices in data modelling, ETL processes and performance optimisation … Docker/Kubernetes) - experience in this area is desirable, not essential Operate within Agile teams and support DevOps practices What We're Looking For Proven experience as a Data Engineer in complex environments Strong proficiency in PostgreSQL and either Airflow or Spark Solid understanding of Node.js or React for integration and tooling Familiarity with containerisation technologies (Docker/ More ❯

and over £20M in annual sales, they're now investing heavily in their technology to unlock the next phase: scaling from 150 sites to 5,000+, powered by data, AI, and automation. The Role You'll be a foundational member of their engineering team. This is a high-impact individual contributor role, ideal for someone who enjoys building … the ground up, solving real-world problems, and owning architecture decisions in a fast-moving environment. You’ll work across the stack, with a focus on backend systems, data infrastructure, and internal tools. What You’ll Be Doing Implementing AI tooling across all their products focusing on data ingestion and aggregation Lead the design and implementation … of scalable, microservices-based systems Own and evolve their core platform architecture, including APIs, internal tools, and data pipelines Work with large, varied data sets from third-party sources (e.g. partner platforms, supply chains, operational data) Help turn data into insights and recommendations using AI/ML tooling Tackle technical debt and More ❯

and over £20M in annual sales, they're now investing heavily in their technology to unlock the next phase: scaling from 150 sites to 5,000+, powered by data, AI, and automation. The Role You'll be a foundational member of their engineering team. This is a high-impact individual contributor role, ideal for someone who enjoys building … the ground up, solving real-world problems, and owning architecture decisions in a fast-moving environment. You’ll work across the stack, with a focus on backend systems, data infrastructure, and internal tools. What You’ll Be Doing Implementing AI tooling across all their products focusing on data ingestion and aggregation Lead the design and implementation … of scalable, microservices-based systems Own and evolve their core platform architecture, including APIs, internal tools, and data pipelines Work with large, varied data sets from third-party sources (e.g. partner platforms, supply chains, operational data) Help turn data into insights and recommendations using AI/ML tooling Tackle technical debt and More ❯

and over £20M in annual sales, they're now investing heavily in their technology to unlock the next phase: scaling from 150 sites to 5,000+, powered by data, AI, and automation. The Role You'll be a foundational member of their engineering team. This is a high-impact individual contributor role, ideal for someone who enjoys building … the ground up, solving real-world problems, and owning architecture decisions in a fast-moving environment. You’ll work across the stack, with a focus on backend systems, data infrastructure, and internal tools. What You’ll Be Doing Implementing AI tooling across all their products focusing on data ingestion and aggregation Lead the design and implementation … of scalable, microservices-based systems Own and evolve their core platform architecture, including APIs, internal tools, and data pipelines Work with large, varied data sets from third-party sources (e.g. partner platforms, supply chains, operational data) Help turn data into insights and recommendations using AI/ML tooling Tackle technical debt and More ❯

and over £20M in annual sales, they're now investing heavily in their technology to unlock the next phase: scaling from 150 sites to 5,000+, powered by data, AI, and automation. The Role You'll be a foundational member of their engineering team. This is a high-impact individual contributor role, ideal for someone who enjoys building … the ground up, solving real-world problems, and owning architecture decisions in a fast-moving environment. You’ll work across the stack, with a focus on backend systems, data infrastructure, and internal tools. What You’ll Be Doing Implementing AI tooling across all their products focusing on data ingestion and aggregation Lead the design and implementation … of scalable, microservices-based systems Own and evolve their core platform architecture, including APIs, internal tools, and data pipelines Work with large, varied data sets from third-party sources (e.g. partner platforms, supply chains, operational data) Help turn data into insights and recommendations using AI/ML tooling Tackle technical debt and More ❯

makers in a live-market environment. Responsibilities: Architect and code resilient applications for trading, liquidity management, and funding operations using Python as the core language, with integration into data platforms and tools. Engage directly with traders and treasury stakeholders during live situations, adapting to evolving requirements and delivering solutions on the fly. Design and optimise data pipelines for real-time and historical treasury data, ensuring accuracy and performance in analytics. Work with operations teams to deploy, monitor, and scale systems, ensuring uptime and quick incident response. Identify technology gaps and inefficiencies, experimenting with new tools and design patterns to enhance speed and quality. Experience: 5-15 years of software engineering experience, ideally … in Treasury or from a hedge fund/buy-side firm Expertise in Python for designing and optimising complex applications Strong proficiency in data tools like Pandas, NumPy, SQL, and real-time data processing Familiarity with DevOps practices, including Linux, cloud platforms, and CI/CD (Docker/Kubernetes is desirable but not essential). Strong More ❯

makers in a live-market environment. Responsibilities: Architect and code resilient applications for trading, liquidity management, and funding operations using Python as the core language, with integration into data platforms and tools. Engage directly with traders and treasury stakeholders during live situations, adapting to evolving requirements and delivering solutions on the fly. Design and optimise data pipelines for real-time and historical treasury data, ensuring accuracy and performance in analytics. Work with operations teams to deploy, monitor, and scale systems, ensuring uptime and quick incident response. Identify technology gaps and inefficiencies, experimenting with new tools and design patterns to enhance speed and quality. Experience: 5-15 years of software engineering experience, ideally … in Treasury or from a hedge fund/buy-side firm Expertise in Python for designing and optimising complex applications Strong proficiency in data tools like Pandas, NumPy, SQL, and real-time data processing Familiarity with DevOps practices, including Linux, cloud platforms, and CI/CD (Docker/Kubernetes is desirable but not essential). Strong More ❯

makers in a live-market environment. Responsibilities: Architect and code resilient applications for trading, liquidity management, and funding operations using Python as the core language, with integration into data platforms and tools. Engage directly with traders and treasury stakeholders during live situations, adapting to evolving requirements and delivering solutions on the fly. Design and optimise data pipelines for real-time and historical treasury data, ensuring accuracy and performance in analytics. Work with operations teams to deploy, monitor, and scale systems, ensuring uptime and quick incident response. Identify technology gaps and inefficiencies, experimenting with new tools and design patterns to enhance speed and quality. Experience: 5-15 years of software engineering experience, ideally … in Treasury or from a hedge fund/buy-side firm Expertise in Python for designing and optimising complex applications Strong proficiency in data tools like Pandas, NumPy, SQL, and real-time data processing Familiarity with DevOps practices, including Linux, cloud platforms, and CI/CD (Docker/Kubernetes is desirable but not essential). Strong More ❯

makers in a live-market environment. Responsibilities: Architect and code resilient applications for trading, liquidity management, and funding operations using Python as the core language, with integration into data platforms and tools. Engage directly with traders and treasury stakeholders during live situations, adapting to evolving requirements and delivering solutions on the fly. Design and optimise data pipelines for real-time and historical treasury data, ensuring accuracy and performance in analytics. Work with operations teams to deploy, monitor, and scale systems, ensuring uptime and quick incident response. Identify technology gaps and inefficiencies, experimenting with new tools and design patterns to enhance speed and quality. Experience: 5-15 years of software engineering experience, ideally … in Treasury or from a hedge fund/buy-side firm Expertise in Python for designing and optimising complex applications Strong proficiency in data tools like Pandas, NumPy, SQL, and real-time data processing Familiarity with DevOps practices, including Linux, cloud platforms, and CI/CD (Docker/Kubernetes is desirable but not essential). Strong More ❯

shape the future of AI. What You’ll Be Doing Research, train, and fine-tune large language models (LLMs) for real-world applications. Develop and optimise pipelines for data collection, preprocessing, and model evaluation. Collaborate with product engineers to deploy models into scalable production systems. Experiment with prompt engineering, RAG architectures, and multimodal models. Contribute to internal tools … ML/AI research — we give you time and resources to explore, learn, and publish. What We’re Looking For 0–3 years of experience in Machine Learning, Data Science, or NLP/LLM. Strong Python skills; exposure to PyTorch/TensorFlow/Hugging Face. (Bonus) understand fundamentals of deep learning, LLMs, and MLOps, vector databases, embeddings, or … creativity, and personal growth over bureaucracy. Perfect For Graduates or junior engineers with a passion for AI/ML looking to break into applied LLM engineering. Researchers or data scientists eager to move from theory to real-world deployment. Builders who want to join an early-stage company where their work genuinely moves the needle. Machine Learning Engineer More ❯

Experienced Technical Consultant with solid experience in the AdTech & MarTech Domain. Strong privacy and consent expertise to lead the design and integration of privacy-centric digital marketing and data activation platforms. You will bridge marketing technology, data governance, privacy compliance, ensuring regulatory requirements and consent principles are embedded in platform design, integration, data flows … of AdTech and MarTech solutions, including CDPs, consent platforms, identity & tagging stacks and activation pipelines. Assess architectures covering tagging, event tracking, attribution, experimentation, audience management and media integrations. Data flows, system interactions and integration patterns across first-party, third-party and cloud-based platforms. Establish guardrails for data use, identity, cookies, measurement, tracking and consent enforcement. … Privacy & Consent Ensure privacy principles (GDPR, PECR, ePrivacy, ICO guidance) and consent controls into architecture and data pipelines. Ensure compliant audience segmentation, targeting, data enrichment, and cross-channel tracking strategies. Guide the implementation of CMPs (Consent Management Platforms), consent signals (TCF), and privacy-centric tagging frameworks. Validate solutions post-deployment to ensure privacy, data More ❯

Experienced Technical Consultant with solid experience in the AdTech & MarTech Domain. Strong privacy and consent expertise to lead the design and integration of privacy-centric digital marketing and data activation platforms. You will bridge marketing technology, data governance, privacy compliance, ensuring regulatory requirements and consent principles are embedded in platform design, integration, data flows … of AdTech and MarTech solutions, including CDPs, consent platforms, identity & tagging stacks and activation pipelines. Assess architectures covering tagging, event tracking, attribution, experimentation, audience management and media integrations. Data flows, system interactions and integration patterns across first-party, third-party and cloud-based platforms. Establish guardrails for data use, identity, cookies, measurement, tracking and consent enforcement. … Privacy & Consent Ensure privacy principles (GDPR, PECR, ePrivacy, ICO guidance) and consent controls into architecture and data pipelines. Ensure compliant audience segmentation, targeting, data enrichment, and cross-channel tracking strategies. Guide the implementation of CMPs (Consent Management Platforms), consent signals (TCF), and privacy-centric tagging frameworks. Validate solutions post-deployment to ensure privacy, data More ❯

Experienced Technical Consultant with solid experience in the AdTech & MarTech Domain. Strong privacy and consent expertise to lead the design and integration of privacy-centric digital marketing and data activation platforms. You will bridge marketing technology, data governance, privacy compliance, ensuring regulatory requirements and consent principles are embedded in platform design, integration, data flows … of AdTech and MarTech solutions, including CDPs, consent platforms, identity & tagging stacks and activation pipelines. Assess architectures covering tagging, event tracking, attribution, experimentation, audience management and media integrations. Data flows, system interactions and integration patterns across first-party, third-party and cloud-based platforms. Establish guardrails for data use, identity, cookies, measurement, tracking and consent enforcement. … Privacy & Consent Ensure privacy principles (GDPR, PECR, ePrivacy, ICO guidance) and consent controls into architecture and data pipelines. Ensure compliant audience segmentation, targeting, data enrichment, and cross-channel tracking strategies. Guide the implementation of CMPs (Consent Management Platforms), consent signals (TCF), and privacy-centric tagging frameworks. Validate solutions post-deployment to ensure privacy, data More ❯

Experienced Technical Consultant with solid experience in the AdTech & MarTech Domain. Strong privacy and consent expertise to lead the design and integration of privacy-centric digital marketing and data activation platforms. You will bridge marketing technology, data governance, privacy compliance, ensuring regulatory requirements and consent principles are embedded in platform design, integration, data flows … of AdTech and MarTech solutions, including CDPs, consent platforms, identity & tagging stacks and activation pipelines. Assess architectures covering tagging, event tracking, attribution, experimentation, audience management and media integrations. Data flows, system interactions and integration patterns across first-party, third-party and cloud-based platforms. Establish guardrails for data use, identity, cookies, measurement, tracking and consent enforcement. … Privacy & Consent Ensure privacy principles (GDPR, PECR, ePrivacy, ICO guidance) and consent controls into architecture and data pipelines. Ensure compliant audience segmentation, targeting, data enrichment, and cross-channel tracking strategies. Guide the implementation of CMPs (Consent Management Platforms), consent signals (TCF), and privacy-centric tagging frameworks. Validate solutions post-deployment to ensure privacy, data More ❯

Databricks Tech Lead – Azure Data Engineering (Contract) We are looking for a highly technical, hands-on Databricks Tech Lead to join a large-scale digital transformation programme within the Energy Trading space. This role will focus on building, optimising, and scaling a data lake on Azure with particular emphasis on streaming pipelines, Delta Live Tables (DLT … and cost-efficient architecture. Role: Designing and developing streaming and batch data pipelines in Databricks (including DLT). Managing structured streaming in continuous mode, ensuring low-latency data delivery. Collaborating with architects, consultants, and business stakeholders to build a unified Azure-based lakehouse. Troubleshooting and optimising performance bottlenecks in high-volume streaming systems. Applying best practices … for cloud cost management and efficient Databricks resource usage. Integrating data from on-prem and cloud sources using tools like Qlik Replicate (CDC). Supporting future phases involving data science and machine learning enablement. Tech Environment Platform: Azure Core Tools: Databricks, Delta Live Tables, Qlik Replicate (CDC) Architecture: Hybrid (on-prem + cloud) Data More ❯

Databricks Tech Lead – Azure Data Engineering (Contract) We are looking for a highly technical, hands-on Databricks Tech Lead to join a large-scale digital transformation programme within the Energy Trading space. This role will focus on building, optimising, and scaling a data lake on Azure with particular emphasis on streaming pipelines, Delta Live Tables (DLT … and cost-efficient architecture. Role: Designing and developing streaming and batch data pipelines in Databricks (including DLT). Managing structured streaming in continuous mode, ensuring low-latency data delivery. Collaborating with architects, consultants, and business stakeholders to build a unified Azure-based lakehouse. Troubleshooting and optimising performance bottlenecks in high-volume streaming systems. Applying best practices … for cloud cost management and efficient Databricks resource usage. Integrating data from on-prem and cloud sources using tools like Qlik Replicate (CDC). Supporting future phases involving data science and machine learning enablement. Tech Environment Platform: Azure Core Tools: Databricks, Delta Live Tables, Qlik Replicate (CDC) Architecture: Hybrid (on-prem + cloud) Data More ❯

Databricks Tech Lead – Azure Data Engineering (Contract) We are looking for a highly technical, hands-on Databricks Tech Lead to join a large-scale digital transformation programme within the Energy Trading space. This role will focus on building, optimising, and scaling a data lake on Azure with particular emphasis on streaming pipelines, Delta Live Tables (DLT … and cost-efficient architecture. Role: Designing and developing streaming and batch data pipelines in Databricks (including DLT). Managing structured streaming in continuous mode, ensuring low-latency data delivery. Collaborating with architects, consultants, and business stakeholders to build a unified Azure-based lakehouse. Troubleshooting and optimising performance bottlenecks in high-volume streaming systems. Applying best practices … for cloud cost management and efficient Databricks resource usage. Integrating data from on-prem and cloud sources using tools like Qlik Replicate (CDC). Supporting future phases involving data science and machine learning enablement. Tech Environment Platform: Azure Core Tools: Databricks, Delta Live Tables, Qlik Replicate (CDC) Architecture: Hybrid (on-prem + cloud) Data More ❯

Databricks Tech Lead – Azure Data Engineering (Contract) We are looking for a highly technical, hands-on Databricks Tech Lead to join a large-scale digital transformation programme within the Energy Trading space. This role will focus on building, optimising, and scaling a data lake on Azure with particular emphasis on streaming pipelines, Delta Live Tables (DLT … and cost-efficient architecture. Role: Designing and developing streaming and batch data pipelines in Databricks (including DLT). Managing structured streaming in continuous mode, ensuring low-latency data delivery. Collaborating with architects, consultants, and business stakeholders to build a unified Azure-based lakehouse. Troubleshooting and optimising performance bottlenecks in high-volume streaming systems. Applying best practices … for cloud cost management and efficient Databricks resource usage. Integrating data from on-prem and cloud sources using tools like Qlik Replicate (CDC). Supporting future phases involving data science and machine learning enablement. Tech Environment Platform: Azure Core Tools: Databricks, Delta Live Tables, Qlik Replicate (CDC) Architecture: Hybrid (on-prem + cloud) Data More ❯

shape our culture and guide our decision-making. We are committed to: Excellence Responsibility Empowerment Connectivity Courage Role Overview We are seeking a seasoned and visionary Lead, AI & Data Science to drive the strategy, development, and deployment of advanced AI solutions across our Digital Solutions department. This role is leadership and technically grounded, you will define and guide … and the ability to review, advise, and step in when critical decisions or mentorship are needed. You will play a pivotal role in building and scaling the AI & Data Science team, shaping career paths, and representing the department in global events, partnerships, and thought leadership platforms. Key Responsibilities Strategic & Technical Leadership Define and lead the AI & Data … CI/CD for ML, monitoring, retraining, governance). Drive innovation while balancing pragmatism and production readiness . Mentorship & Team Development Build and grow a world-class AI & Data Science team , including hiring, onboarding, and performance management . Mentor and coach team members to elevate technical depth and problem-solving skills. Create career development plans, learning paths, and More ❯

shape our culture and guide our decision-making. We are committed to: Excellence Responsibility Empowerment Connectivity Courage Role Overview We are seeking a seasoned and visionary Lead, AI & Data Science to drive the strategy, development, and deployment of advanced AI solutions across our Digital Solutions department. This role is leadership and technically grounded, you will define and guide … and the ability to review, advise, and step in when critical decisions or mentorship are needed. You will play a pivotal role in building and scaling the AI & Data Science team, shaping career paths, and representing the department in global events, partnerships, and thought leadership platforms. Key Responsibilities Strategic & Technical Leadership Define and lead the AI & Data … CI/CD for ML, monitoring, retraining, governance). Drive innovation while balancing pragmatism and production readiness . Mentorship & Team Development Build and grow a world-class AI & Data Science team , including hiring, onboarding, and performance management . Mentor and coach team members to elevate technical depth and problem-solving skills. Create career development plans, learning paths, and More ❯

The Role As AI Reporting & Insights Lead you'll own the intelligence layer of our AI-powered platform—the dashboards, reports, prompts, and templates that transform raw customer data into clarity. This is a unique hybrid role combining AI expertise, analytical rigor, and customer-facing delivery. You'll define what "great AI-powered reporting" looks like and make … features Close the feedback loop: Channel customer needs back to Product to shape the reporting roadmap Product Collaboration & Innovation (20%) Shape AI reporting features: Partner with Product and Data Science to spec new capabilities—structured output schemas, prompt templates, embedding-based discovery Experiment & prototype: Build proof-of-concepts for new report types using Python, pandas, OpenAI APIs, and … our data pipelines QA & iteration: Test new features, validate AI outputs for accuracy and relevance, ensure quality before launch Operations & Optimization (10%) Monitor report health: Track delivery, engagement, and quality metrics; debug when outputs degrade Performance tuning: Optimize report generation costs and latency; balance API usage with quality Document everything: Maintain clear documentation for prompts, templates, and best More ❯

The Role As AI Reporting & Insights Lead you'll own the intelligence layer of our AI-powered platform—the dashboards, reports, prompts, and templates that transform raw customer data into clarity. This is a unique hybrid role combining AI expertise, analytical rigor, and customer-facing delivery. You'll define what "great AI-powered reporting" looks like and make … features Close the feedback loop: Channel customer needs back to Product to shape the reporting roadmap Product Collaboration & Innovation (20%) Shape AI reporting features: Partner with Product and Data Science to spec new capabilities—structured output schemas, prompt templates, embedding-based discovery Experiment & prototype: Build proof-of-concepts for new report types using Python, pandas, OpenAI APIs, and … our data pipelines QA & iteration: Test new features, validate AI outputs for accuracy and relevance, ensure quality before launch Operations & Optimization (10%) Monitor report health: Track delivery, engagement, and quality metrics; debug when outputs degrade Performance tuning: Optimize report generation costs and latency; balance API usage with quality Document everything: Maintain clear documentation for prompts, templates, and best More ❯

The Role As AI Reporting & Insights Lead you'll own the intelligence layer of our AI-powered platform—the dashboards, reports, prompts, and templates that transform raw customer data into clarity. This is a unique hybrid role combining AI expertise, analytical rigor, and customer-facing delivery. You'll define what "great AI-powered reporting" looks like and make … features Close the feedback loop: Channel customer needs back to Product to shape the reporting roadmap Product Collaboration & Innovation (20%) Shape AI reporting features: Partner with Product and Data Science to spec new capabilities—structured output schemas, prompt templates, embedding-based discovery Experiment & prototype: Build proof-of-concepts for new report types using Python, pandas, OpenAI APIs, and … our data pipelines QA & iteration: Test new features, validate AI outputs for accuracy and relevance, ensure quality before launch Operations & Optimization (10%) Monitor report health: Track delivery, engagement, and quality metrics; debug when outputs degrade Performance tuning: Optimize report generation costs and latency; balance API usage with quality Document everything: Maintain clear documentation for prompts, templates, and best More ❯

The Role As AI Reporting & Insights Lead you'll own the intelligence layer of our AI-powered platform—the dashboards, reports, prompts, and templates that transform raw customer data into clarity. This is a unique hybrid role combining AI expertise, analytical rigor, and customer-facing delivery. You'll define what "great AI-powered reporting" looks like and make … features Close the feedback loop: Channel customer needs back to Product to shape the reporting roadmap Product Collaboration & Innovation (20%) Shape AI reporting features: Partner with Product and Data Science to spec new capabilities—structured output schemas, prompt templates, embedding-based discovery Experiment & prototype: Build proof-of-concepts for new report types using Python, pandas, OpenAI APIs, and … our data pipelines QA & iteration: Test new features, validate AI outputs for accuracy and relevance, ensure quality before launch Operations & Optimization (10%) Monitor report health: Track delivery, engagement, and quality metrics; debug when outputs degrade Performance tuning: Optimize report generation costs and latency; balance API usage with quality Document everything: Maintain clear documentation for prompts, templates, and best More ❯