826 to 850 of 1,063 Data Pipeline Jobs in London

Engineer - London (Hybrid) - Up to £90k A well funded start up who are revolutionising real estate transactions through cutting-edge technology combining sophisticated AI capabilities with robust financial data systems are looking for a Senior Full Stack Engineer to join their team. What will I be doing: Design, develop, test, deploy, and maintain robust and scalable backend systems … Work with databases to design schemas, write efficient queries, and ensure optimal performance Contribute to overall system architecture, ensuring scalability, reliability, and security Integrate various platform components, including data pipelines and AI-driven features Mentor junior engineers and contribute to improving our engineering practices and processes What are we looking for: At least 5+ years in professional full … stack software development Strong proficiency in Python for backend development Experience with modern frontend frameworks (React, Vue, Angular) Solid understanding of database technologies (PostgreSQL, MySQL) and data modeling Experience designing and building RESTful APIs Experience with cloud platforms (AWS, GCP, Azure) and containerisation (Docker) Bonus: Experience with financial concepts, real estate data, or startup environments What More ❯

major national programme. This is a fantastic opportunity to take ownership of building and leading a team delivering cutting-edge digital infrastructure that will transform how vital scientific data is accessed and shared across the UK and beyond. The role As Lead Developer, you’ll provide senior technical direction and hands-on expertise in designing and implementing scalable … digital infrastructure programme. Provide hands-on technical input, writing high-quality code and driving architecture decisions. Design and implement secure, scalable backend systems and cloud infrastructure. Build robust data pipelines to manage and mobilise large-scale datasets. Collaborate with external suppliers, partners, and stakeholders to align delivery. What we’re looking for: At least 5 years of experience More ❯

major national programme. This is a fantastic opportunity to take ownership of building and leading a team delivering cutting-edge digital infrastructure that will transform how vital scientific data is accessed and shared across the UK and beyond. The role As Lead Developer, you’ll provide senior technical direction and hands-on expertise in designing and implementing scalable … digital infrastructure programme. Provide hands-on technical input, writing high-quality code and driving architecture decisions. Design and implement secure, scalable backend systems and cloud infrastructure. Build robust data pipelines to manage and mobilise large-scale datasets. Collaborate with external suppliers, partners, and stakeholders to align delivery. What we’re looking for: At least 5 years of experience More ❯

major national programme. This is a fantastic opportunity to take ownership of building and leading a team delivering cutting-edge digital infrastructure that will transform how vital scientific data is accessed and shared across the UK and beyond. The role As Lead Developer, you’ll provide senior technical direction and hands-on expertise in designing and implementing scalable … digital infrastructure programme. Provide hands-on technical input, writing high-quality code and driving architecture decisions. Design and implement secure, scalable backend systems and cloud infrastructure. Build robust data pipelines to manage and mobilise large-scale datasets. Collaborate with external suppliers, partners, and stakeholders to align delivery. What we’re looking for: At least 5 years of experience More ❯

major national programme. This is a fantastic opportunity to take ownership of building and leading a team delivering cutting-edge digital infrastructure that will transform how vital scientific data is accessed and shared across the UK and beyond. The role As Lead Developer, you’ll provide senior technical direction and hands-on expertise in designing and implementing scalable … digital infrastructure programme. Provide hands-on technical input, writing high-quality code and driving architecture decisions. Design and implement secure, scalable backend systems and cloud infrastructure. Build robust data pipelines to manage and mobilise large-scale datasets. Collaborate with external suppliers, partners, and stakeholders to align delivery. What we’re looking for: At least 5 years of experience More ❯

AWS, GCP, Azure) for ML workloads and APIs Setting up ML nodes for distributed training and local development Managing containerised environments (Docker, Kubernetes, Terraform) Optimising storage for big data pipelines supporting ML workloads Monitoring systems and responding to incidents, ensuring reliability and performance Working closely with ML engineers and researchers to integrate infra with production workloads What you … in automating deployments and orchestrating cloud environments Nice to have: Python (Jupyter, PyTorch), monitoring tools (Prometheus, Grafana), cloud databases (RDS, Aurora, Spanner), CI/CD tools (CircleCI), and data visualisation experience. This is a unique opportunity to join a visionary team redefining AI in 3D , with the chance to make a real impact at the cutting edge of More ❯

AWS, GCP, Azure) for ML workloads and APIs Setting up ML nodes for distributed training and local development Managing containerised environments (Docker, Kubernetes, Terraform) Optimising storage for big data pipelines supporting ML workloads Monitoring systems and responding to incidents, ensuring reliability and performance Working closely with ML engineers and researchers to integrate infra with production workloads What you … in automating deployments and orchestrating cloud environments Nice to have: Python (Jupyter, PyTorch), monitoring tools (Prometheus, Grafana), cloud databases (RDS, Aurora, Spanner), CI/CD tools (CircleCI), and data visualisation experience. This is a unique opportunity to join a visionary team redefining AI in 3D , with the chance to make a real impact at the cutting edge of More ❯

be doing: Lead the design and deployment of next-generation AI systems across Generative AI, Computer Vision, Forecasting, and Optimisation. Build and lead a world-class team of data scientists and ML engineers, setting technical direction and best practices across the company. Oversee the full AI lifecycle from experimentation and model development to robust ML deployment on Azure. … learning to enhance decision-making and automation. Translate complex research into tangible business outcomes, driving measurable value and competitive advantage. About you: Proven track record leading AI or data science teams in production supply chain or logistics environments. Deep expertise in Computer Vision and Generative AI, with strong understanding of modern ML architectures, model training, and deployment techniques. … 7+ years’ experience in applied ML, data science, or AI, including senior or leadership roles. Expert in Python, proficient with frameworks such as PyTorch, TensorFlow, FastAI, and experience with scikit-learn, MLflow, or ONNX. Solid engineering background—experienced in deploying and maintaining ML systems on Azure, including Azure Machine Learning, Azure Functions, Databricks, Docker, and Kubernetes. Experience with More ❯

be doing: Lead the design and deployment of next-generation AI systems across Generative AI, Computer Vision, Forecasting, and Optimisation. Build and lead a world-class team of data scientists and ML engineers, setting technical direction and best practices across the company. Oversee the full AI lifecycle from experimentation and model development to robust ML deployment on Azure. … learning to enhance decision-making and automation. Translate complex research into tangible business outcomes, driving measurable value and competitive advantage. About you: Proven track record leading AI or data science teams in production supply chain or logistics environments. Deep expertise in Computer Vision and Generative AI, with strong understanding of modern ML architectures, model training, and deployment techniques. … 7+ years’ experience in applied ML, data science, or AI, including senior or leadership roles. Expert in Python, proficient with frameworks such as PyTorch, TensorFlow, FastAI, and experience with scikit-learn, MLflow, or ONNX. Solid engineering background—experienced in deploying and maintaining ML systems on Azure, including Azure Machine Learning, Azure Functions, Databricks, Docker, and Kubernetes. Experience with More ❯

be doing: Lead the design and deployment of next-generation AI systems across Generative AI, Computer Vision, Forecasting, and Optimisation. Build and lead a world-class team of data scientists and ML engineers, setting technical direction and best practices across the company. Oversee the full AI lifecycle from experimentation and model development to robust ML deployment on Azure. … learning to enhance decision-making and automation. Translate complex research into tangible business outcomes, driving measurable value and competitive advantage. About you: Proven track record leading AI or data science teams in production supply chain or logistics environments. Deep expertise in Computer Vision and Generative AI, with strong understanding of modern ML architectures, model training, and deployment techniques. … 7+ years’ experience in applied ML, data science, or AI, including senior or leadership roles. Expert in Python, proficient with frameworks such as PyTorch, TensorFlow, FastAI, and experience with scikit-learn, MLflow, or ONNX. Solid engineering background—experienced in deploying and maintaining ML systems on Azure, including Azure Machine Learning, Azure Functions, Databricks, Docker, and Kubernetes. Experience with More ❯

be doing: Lead the design and deployment of next-generation AI systems across Generative AI, Computer Vision, Forecasting, and Optimisation. Build and lead a world-class team of data scientists and ML engineers, setting technical direction and best practices across the company. Oversee the full AI lifecycle from experimentation and model development to robust ML deployment on Azure. … learning to enhance decision-making and automation. Translate complex research into tangible business outcomes, driving measurable value and competitive advantage. About you: Proven track record leading AI or data science teams in production supply chain or logistics environments. Deep expertise in Computer Vision and Generative AI, with strong understanding of modern ML architectures, model training, and deployment techniques. … 7+ years’ experience in applied ML, data science, or AI, including senior or leadership roles. Expert in Python, proficient with frameworks such as PyTorch, TensorFlow, FastAI, and experience with scikit-learn, MLflow, or ONNX. Solid engineering background—experienced in deploying and maintaining ML systems on Azure, including Azure Machine Learning, Azure Functions, Databricks, Docker, and Kubernetes. Experience with More ❯

Lead Data Engineer - Python, SQL, GCP, Kubernetes, DBT, Airflow and BigQurey OB have partnered with a leading scale-up business in the AdTech space, where Data is at the forefront of everything they do, and they are currently hiring for a Lead Data Engineer to join their team and work on the development of … their Data platform. This will be a hands-on role but you will line manage a small team of 3 Engineers, where the long-term plan of this team is to grow and this is a great opportunity to progress in a scaling business. This role requires candidates to come with hands on experience working in Data heavy environments, with Real-Time Data Pipelines, Distributed Streaming Pipelines and strong knowledge of Cloud Environments. Lead Data Engineer - Python, SQL, GCP, Kubernetes, DBT, Airflow and BigQurey Key Skills and Experience: Python, SQL, GCP, Kubernetes, DBT, Airflow and BigQurey Prior experience working as a Lead Engineer or Tech Lead Pays £100k-£130k + bonus More ❯

Lead Data Engineer - Python, SQL, GCP, Kubernetes, DBT, Airflow and BigQurey OB have partnered with a leading scale-up business in the AdTech space, where Data is at the forefront of everything they do, and they are currently hiring for a Lead Data Engineer to join their team and work on the development of … their Data platform. This will be a hands-on role but you will line manage a small team of 3 Engineers, where the long-term plan of this team is to grow and this is a great opportunity to progress in a scaling business. This role requires candidates to come with hands on experience working in Data heavy environments, with Real-Time Data Pipelines, Distributed Streaming Pipelines and strong knowledge of Cloud Environments. Lead Data Engineer - Python, SQL, GCP, Kubernetes, DBT, Airflow and BigQurey Key Skills and Experience: Python, SQL, GCP, Kubernetes, DBT, Airflow and BigQurey Prior experience working as a Lead Engineer or Tech Lead Pays £100k-£130k + bonus More ❯

Position: Data EngineerLocation: Central London or Bristol (3 days per week onsite)Salary: £600 to £650 p.d inside IR35Security Clearance: SC Clearance Required About the Role A leading IT consultancy delivering innovative solutions to the UK public sector is seeking an experienced Data Engineer to support a high-profile programme. You'll collaborate with cross-functional … teams to build scalable data pipelines and contribute to digital transformation initiatives across government departments. Key Responsibilities Design, develop and maintain robust data pipelines using PostgreSQL and Airflow or Apache Spark Collaborate with frontend/backend developers using Node.js or React Implement best practices in data modelling, ETL processes and performance optimisation Contribute to … Docker/Kubernetes) - experience in this area is desirable, not essential Operate within Agile teams and support DevOps practices What We're Looking For Proven experience as a Data Engineer in complex environments Strong proficiency in PostgreSQL and either Airflow or Spark Solid understanding of Node.js or React for integration and tooling Familiarity with containerisation technologies (Docker/ More ❯

Position: Data Engineer Location: Central London or Bristol (3 days per week onsite) Salary: £600 to £650 p.d inside IR35 Security Clearance: SC Clearance Required About the Role A leading IT consultancy delivering innovative solutions to the UK public sector is seeking an experienced Data Engineer to support a high-profile programme. You'll collaborate with … cross-functional teams to build scalable data pipelines and contribute to digital transformation initiatives across government departments. Key Responsibilities Design, develop and maintain robust data pipelines using PostgreSQL and Airflow or Apache Spark Collaborate with frontend/backend developers using Node.js or React Implement best practices in data modelling, ETL processes and performance optimisation … Docker/Kubernetes) - experience in this area is desirable, not essential Operate within Agile teams and support DevOps practices What We're Looking For Proven experience as a Data Engineer in complex environments Strong proficiency in PostgreSQL and either Airflow or Spark Solid understanding of Node.js or React for integration and tooling Familiarity with containerisation technologies (Docker/ More ❯

and over £20M in annual sales, they're now investing heavily in their technology to unlock the next phase: scaling from 150 sites to 5,000+, powered by data, AI, and automation. The Role You'll be a foundational member of their engineering team. This is a high-impact individual contributor role, ideal for someone who enjoys building … the ground up, solving real-world problems, and owning architecture decisions in a fast-moving environment. You’ll work across the stack, with a focus on backend systems, data infrastructure, and internal tools. What You’ll Be Doing Implementing AI tooling across all their products focusing on data ingestion and aggregation Lead the design and implementation … of scalable, microservices-based systems Own and evolve their core platform architecture, including APIs, internal tools, and data pipelines Work with large, varied data sets from third-party sources (e.g. partner platforms, supply chains, operational data) Help turn data into insights and recommendations using AI/ML tooling Tackle technical debt and More ❯

and over £20M in annual sales, they're now investing heavily in their technology to unlock the next phase: scaling from 150 sites to 5,000+, powered by data, AI, and automation. The Role You'll be a foundational member of their engineering team. This is a high-impact individual contributor role, ideal for someone who enjoys building … the ground up, solving real-world problems, and owning architecture decisions in a fast-moving environment. You’ll work across the stack, with a focus on backend systems, data infrastructure, and internal tools. What You’ll Be Doing Implementing AI tooling across all their products focusing on data ingestion and aggregation Lead the design and implementation … of scalable, microservices-based systems Own and evolve their core platform architecture, including APIs, internal tools, and data pipelines Work with large, varied data sets from third-party sources (e.g. partner platforms, supply chains, operational data) Help turn data into insights and recommendations using AI/ML tooling Tackle technical debt and More ❯

and over £20M in annual sales, they're now investing heavily in their technology to unlock the next phase: scaling from 150 sites to 5,000+, powered by data, AI, and automation. The Role You'll be a foundational member of their engineering team. This is a high-impact individual contributor role, ideal for someone who enjoys building … the ground up, solving real-world problems, and owning architecture decisions in a fast-moving environment. You’ll work across the stack, with a focus on backend systems, data infrastructure, and internal tools. What You’ll Be Doing Implementing AI tooling across all their products focusing on data ingestion and aggregation Lead the design and implementation … of scalable, microservices-based systems Own and evolve their core platform architecture, including APIs, internal tools, and data pipelines Work with large, varied data sets from third-party sources (e.g. partner platforms, supply chains, operational data) Help turn data into insights and recommendations using AI/ML tooling Tackle technical debt and More ❯

and over £20M in annual sales, they're now investing heavily in their technology to unlock the next phase: scaling from 150 sites to 5,000+, powered by data, AI, and automation. The Role You'll be a foundational member of their engineering team. This is a high-impact individual contributor role, ideal for someone who enjoys building … the ground up, solving real-world problems, and owning architecture decisions in a fast-moving environment. You’ll work across the stack, with a focus on backend systems, data infrastructure, and internal tools. What You’ll Be Doing Implementing AI tooling across all their products focusing on data ingestion and aggregation Lead the design and implementation … of scalable, microservices-based systems Own and evolve their core platform architecture, including APIs, internal tools, and data pipelines Work with large, varied data sets from third-party sources (e.g. partner platforms, supply chains, operational data) Help turn data into insights and recommendations using AI/ML tooling Tackle technical debt and More ❯

makers in a live-market environment. Responsibilities: Architect and code resilient applications for trading, liquidity management, and funding operations using Python as the core language, with integration into data platforms and tools. Engage directly with traders and treasury stakeholders during live situations, adapting to evolving requirements and delivering solutions on the fly. Design and optimise data pipelines for real-time and historical treasury data, ensuring accuracy and performance in analytics. Work with operations teams to deploy, monitor, and scale systems, ensuring uptime and quick incident response. Identify technology gaps and inefficiencies, experimenting with new tools and design patterns to enhance speed and quality. Experience: 5-15 years of software engineering experience, ideally … in Treasury or from a hedge fund/buy-side firm Expertise in Python for designing and optimising complex applications Strong proficiency in data tools like Pandas, NumPy, SQL, and real-time data processing Familiarity with DevOps practices, including Linux, cloud platforms, and CI/CD (Docker/Kubernetes is desirable but not essential). Strong More ❯

makers in a live-market environment. Responsibilities: Architect and code resilient applications for trading, liquidity management, and funding operations using Python as the core language, with integration into data platforms and tools. Engage directly with traders and treasury stakeholders during live situations, adapting to evolving requirements and delivering solutions on the fly. Design and optimise data pipelines for real-time and historical treasury data, ensuring accuracy and performance in analytics. Work with operations teams to deploy, monitor, and scale systems, ensuring uptime and quick incident response. Identify technology gaps and inefficiencies, experimenting with new tools and design patterns to enhance speed and quality. Experience: 5-15 years of software engineering experience, ideally … in Treasury or from a hedge fund/buy-side firm Expertise in Python for designing and optimising complex applications Strong proficiency in data tools like Pandas, NumPy, SQL, and real-time data processing Familiarity with DevOps practices, including Linux, cloud platforms, and CI/CD (Docker/Kubernetes is desirable but not essential). Strong More ❯

makers in a live-market environment. Responsibilities: Architect and code resilient applications for trading, liquidity management, and funding operations using Python as the core language, with integration into data platforms and tools. Engage directly with traders and treasury stakeholders during live situations, adapting to evolving requirements and delivering solutions on the fly. Design and optimise data pipelines for real-time and historical treasury data, ensuring accuracy and performance in analytics. Work with operations teams to deploy, monitor, and scale systems, ensuring uptime and quick incident response. Identify technology gaps and inefficiencies, experimenting with new tools and design patterns to enhance speed and quality. Experience: 5-15 years of software engineering experience, ideally … in Treasury or from a hedge fund/buy-side firm Expertise in Python for designing and optimising complex applications Strong proficiency in data tools like Pandas, NumPy, SQL, and real-time data processing Familiarity with DevOps practices, including Linux, cloud platforms, and CI/CD (Docker/Kubernetes is desirable but not essential). Strong More ❯

makers in a live-market environment. Responsibilities: Architect and code resilient applications for trading, liquidity management, and funding operations using Python as the core language, with integration into data platforms and tools. Engage directly with traders and treasury stakeholders during live situations, adapting to evolving requirements and delivering solutions on the fly. Design and optimise data pipelines for real-time and historical treasury data, ensuring accuracy and performance in analytics. Work with operations teams to deploy, monitor, and scale systems, ensuring uptime and quick incident response. Identify technology gaps and inefficiencies, experimenting with new tools and design patterns to enhance speed and quality. Experience: 5-15 years of software engineering experience, ideally … in Treasury or from a hedge fund/buy-side firm Expertise in Python for designing and optimising complex applications Strong proficiency in data tools like Pandas, NumPy, SQL, and real-time data processing Familiarity with DevOps practices, including Linux, cloud platforms, and CI/CD (Docker/Kubernetes is desirable but not essential). Strong More ❯

shape the future of AI. What You’ll Be Doing Research, train, and fine-tune large language models (LLMs) for real-world applications. Develop and optimise pipelines for data collection, preprocessing, and model evaluation. Collaborate with product engineers to deploy models into scalable production systems. Experiment with prompt engineering, RAG architectures, and multimodal models. Contribute to internal tools … ML/AI research — we give you time and resources to explore, learn, and publish. What We’re Looking For 0–3 years of experience in Machine Learning, Data Science, or NLP/LLM. Strong Python skills; exposure to PyTorch/TensorFlow/Hugging Face. (Bonus) understand fundamentals of deep learning, LLMs, and MLOps, vector databases, embeddings, or … creativity, and personal growth over bureaucracy. Perfect For Graduates or junior engineers with a passion for AI/ML looking to break into applied LLM engineering. Researchers or data scientists eager to move from theory to real-world deployment. Builders who want to join an early-stage company where their work genuinely moves the needle. Machine Learning Engineer More ❯

Experienced Technical Consultant with solid experience in the AdTech & MarTech Domain. Strong privacy and consent expertise to lead the design and integration of privacy-centric digital marketing and data activation platforms. You will bridge marketing technology, data governance, privacy compliance, ensuring regulatory requirements and consent principles are embedded in platform design, integration, data flows … of AdTech and MarTech solutions, including CDPs, consent platforms, identity & tagging stacks and activation pipelines. Assess architectures covering tagging, event tracking, attribution, experimentation, audience management and media integrations. Data flows, system interactions and integration patterns across first-party, third-party and cloud-based platforms. Establish guardrails for data use, identity, cookies, measurement, tracking and consent enforcement. … Privacy & Consent Ensure privacy principles (GDPR, PECR, ePrivacy, ICO guidance) and consent controls into architecture and data pipelines. Ensure compliant audience segmentation, targeting, data enrichment, and cross-channel tracking strategies. Guide the implementation of CMPs (Consent Management Platforms), consent signals (TCF), and privacy-centric tagging frameworks. Validate solutions post-deployment to ensure privacy, data More ❯