276 to 300 of 871 Data Pipeline Jobs in London

Data Engineering Lead – Azure & Databricks | Full Time Role - Hybrid 2/3 days per week London This role has a Flexible salary and company benefits as well get in touch to learn more! My client is looking for a hands-on Data Engineering Lead to shape and evolve a modern hybrid data platform built … on Azure and Databricks. This role is central to enabling scalable, high-performance data solutions that drive analytics and reporting across the organisation. What does the role consist of? Design and enhance data pipelines that support large-scale ingestion, transformation, and delivery across varied sources. Leverage Azure and Databricks to ensure reliable, high-throughput data movement and integration for analytics and BI. Apply best practices in orchestration, monitoring, and error handling to maintain consistent and dependable data flows. Optimise hybrid cloud architecture for performance, scalability, and cost-efficiency. Implement smart storage strategies, compute resource allocation, and caching techniques to support growing data volumes. Lead and mentor a team of More ❯

Fimador is currently recruiting for a Principal Data Engineer to join a team developing cutting-edge … data-driven products. This role is helping the firm pioneer AI and ML in the global environment. This role is central to designing and maintaining scalable pipelines, data platforms, and integrations, while ensuring solutions meet regulatory standards and align with architectural best practices. Key Responsibilities: Build and optimise scalable data pipelines using Databricks and … backlog refinement. Promote engineering best practices including CI/CD, code reviews, and testing. Research and introduce new tools, technologies, and methodologies. Ideal experience: Experience with efficient, reliable data pipelines that improve time-to-insight. Knowledge of secure, auditable, and compliant data workflows. Know how on optimising performance and reducing costs through Spark and Databricks tuning. More ❯

Fimador is currently recruiting for a Principal Data Engineer to join a team developing cutting-edge … data-driven products. This role is helping the firm pioneer AI and ML in the global environment. This role is central to designing and maintaining scalable pipelines, data platforms, and integrations, while ensuring solutions meet regulatory standards and align with architectural best practices. Key Responsibilities: Build and optimise scalable data pipelines using Databricks and … backlog refinement. Promote engineering best practices including CI/CD, code reviews, and testing. Research and introduce new tools, technologies, and methodologies. Ideal experience: Experience with efficient, reliable data pipelines that improve time-to-insight. Knowledge of secure, auditable, and compliant data workflows. Know how on optimising performance and reducing costs through Spark and Databricks tuning. More ❯

Fimador is currently recruiting for a Principal Data Engineer to join a team developing cutting-edge … data-driven products. This role is helping the firm pioneer AI and ML in the global environment. This role is central to designing and maintaining scalable pipelines, data platforms, and integrations, while ensuring solutions meet regulatory standards and align with architectural best practices. Key Responsibilities: Build and optimise scalable data pipelines using Databricks and … backlog refinement. Promote engineering best practices including CI/CD, code reviews, and testing. Research and introduce new tools, technologies, and methodologies. Ideal experience: Experience with efficient, reliable data pipelines that improve time-to-insight. Knowledge of secure, auditable, and compliant data workflows. Know how on optimising performance and reducing costs through Spark and Databricks tuning. More ❯

Fimador is currently recruiting for a Principal Data Engineer to join a team developing cutting-edge … data-driven products. This role is helping the firm pioneer AI and ML in the global environment. This role is central to designing and maintaining scalable pipelines, data platforms, and integrations, while ensuring solutions meet regulatory standards and align with architectural best practices. Key Responsibilities: Build and optimise scalable data pipelines using Databricks and … backlog refinement. Promote engineering best practices including CI/CD, code reviews, and testing. Research and introduce new tools, technologies, and methodologies. Ideal experience: Experience with efficient, reliable data pipelines that improve time-to-insight. Knowledge of secure, auditable, and compliant data workflows. Know how on optimising performance and reducing costs through Spark and Databricks tuning. More ❯

Senior Data & ML Engineer? Cardiff/London? £90,000 - £100,000 DOE What You'll Do: Take the lead on building and scaling large-scale web scraping and data ingestion systems that form the core of an AI-driven product suite used by major enterprise clients. From crawling hundreds of dynamic sources to powering downstream LLM … features, your work will be the foundation of everything the platform delivers. You'll own projects end-to-end - designing data pipelines, integrating with vector databases, and deploying intelligent features using Retrieval-Augmented Generation (RAG) and fine-tuned LLMs. What You'll Bring: Deep hands-on experience with web scraping tools and frameworks (Scrapy, Playwright, Puppeteer, etc.) 5+ … Strong Python skills and comfort working across complex ingestion workflows Experience managing NoSQL and vector databases at scale (MongoDB, Weaviate, Pinecone, etc.) Solid understanding of modern data pipeline tools (Airflow, Prefect, Dagster) Practical experience with LLM development, embeddings, and RAG architectures Familiarity with distributed systems and cloud platforms (AWS, GCP, or Azure) Self-motivated and capable of More ❯

Data Engineer – Hedge Fund | AWS | SQL | Python | Power BI Location: London Contract: Full-time Salary: £150,000 + Bonus + Benefits A leading hedge fund is seeking a skilled Data Engineer to join its front-office technology team. This is a high-impact role where you’ll work directly with portfolio managers, quants, and analysts to … design and deliver scalable, cloud-based data solutions that power trading and investment strategies. Key Responsibilities Design, build, and optimise data pipelines and ETL workflows using AWS, Python, and SQL. Develop and maintain data models, ensuring accuracy and reliability of trading and market data. Deliver Power BI dashboards and reports to provide real-time … insights for investment teams. Partner with stakeholders across the equities business to understand data requirements and deliver impactful solutions. Ensure data quality, governance, and performance standards are consistently met. Collaborate with quants, traders, and developers to support research, trading, and risk workflows. What you’ll bring Proven experience as a Data Engineer within financial More ❯

Data Engineer – Hedge Fund | AWS | SQL | Python | Power BI Location: London Contract: Full-time Salary: £150,000 + Bonus + Benefits A leading hedge fund is seeking a skilled Data Engineer to join its front-office technology team. This is a high-impact role where you’ll work directly with portfolio managers, quants, and analysts to … design and deliver scalable, cloud-based data solutions that power trading and investment strategies. Key Responsibilities Design, build, and optimise data pipelines and ETL workflows using AWS, Python, and SQL. Develop and maintain data models, ensuring accuracy and reliability of trading and market data. Deliver Power BI dashboards and reports to provide real-time … insights for investment teams. Partner with stakeholders across the equities business to understand data requirements and deliver impactful solutions. Ensure data quality, governance, and performance standards are consistently met. Collaborate with quants, traders, and developers to support research, trading, and risk workflows. What you’ll bring Proven experience as a Data Engineer within financial More ❯

Data & Technology Lead London (3 days in Office) | Up to £70,000 + Bonus I’m working with an employee-owned media agency that’s looking for a hands-on Data & Technology Lead to take ownership of their data engineering and automation capabilities. This role will focus on building and optimising data pipelines, shaping data architecture, and driving the use of AI and automation across the business. You’ll also mentor a small team and play a key part in developing the agency’s future data strategy. What you’ll bring: Strong background in data engineering – Python, SQL, and cloud platforms (GCP or Azure … . Experience designing or managing data pipelines and ETL processes . Exposure to orchestration tools such as dbt , Airflow , or Azure Data Factory . Good understanding of data architecture and automation . AdTech or MarTech experience would be a real plus. Confident mentoring and developing junior engineers/analysts. This is a great opportunity More ❯

Data & Technology Lead London (3 days in Office) | Up to £70,000 + Bonus I’m working with an employee-owned media agency that’s looking for a hands-on Data & Technology Lead to take ownership of their data engineering and automation capabilities. This role will focus on building and optimising data pipelines, shaping data architecture, and driving the use of AI and automation across the business. You’ll also mentor a small team and play a key part in developing the agency’s future data strategy. What you’ll bring: Strong background in data engineering – Python, SQL, and cloud platforms (GCP or Azure … . Experience designing or managing data pipelines and ETL processes . Exposure to orchestration tools such as dbt , Airflow , or Azure Data Factory . Good understanding of data architecture and automation . AdTech or MarTech experience would be a real plus. Confident mentoring and developing junior engineers/analysts. This is a great opportunity More ❯

Data & Technology Lead London (3 days in Office) | Up to £70,000 + Bonus I’m working with an employee-owned media agency that’s looking for a hands-on Data & Technology Lead to take ownership of their data engineering and automation capabilities. This role will focus on building and optimising data pipelines, shaping data architecture, and driving the use of AI and automation across the business. You’ll also mentor a small team and play a key part in developing the agency’s future data strategy. What you’ll bring: Strong background in data engineering – Python, SQL, and cloud platforms (GCP or Azure … . Experience designing or managing data pipelines and ETL processes . Exposure to orchestration tools such as dbt , Airflow , or Azure Data Factory . Good understanding of data architecture and automation . AdTech or MarTech experience would be a real plus. Confident mentoring and developing junior engineers/analysts. This is a great opportunity More ❯

Data & Technology Lead London (3 days in Office) | Up to £70,000 + Bonus I’m working with an employee-owned media agency that’s looking for a hands-on Data & Technology Lead to take ownership of their data engineering and automation capabilities. This role will focus on building and optimising data pipelines, shaping data architecture, and driving the use of AI and automation across the business. You’ll also mentor a small team and play a key part in developing the agency’s future data strategy. What you’ll bring: Strong background in data engineering – Python, SQL, and cloud platforms (GCP or Azure … . Experience designing or managing data pipelines and ETL processes . Exposure to orchestration tools such as dbt , Airflow , or Azure Data Factory . Good understanding of data architecture and automation . AdTech or MarTech experience would be a real plus. Confident mentoring and developing junior engineers/analysts. This is a great opportunity More ❯

around £500/day Starting ASAP for 6 months - likely to extend About the Role We’re partnering with a financial client that’s undertaking a large-scale data transformation, modernising legacy systems and building a unified analytics platform in the cloud. As part of this initiative, you’ll play a key role in developing and optimising a … Snowflake-based data warehouse and delivering high-quality visualisations and reporting through Power BI. The role offers the chance to work on greenfield architecture, migrate legacy SQL Server solutions and collaborate closely with senior stakeholders on complex, high-impact data initiatives. What You’ll Be Doing Design, develop and maintain data pipelines and models … using Snowflake, dbt and Python. Enhance and support Power BI dashboards and data models, ensuring performance, accuracy and user accessibility. Contribute to the migration of legacy SQL Server (SSIS/SSAS/SSRS) systems into a modern Snowflake/Power BI environment. Work collaboratively with business users to understand requirements, translate them into technical solutions and deliver robust More ❯

around £500/day Starting ASAP for 6 months - likely to extend About the Role We’re partnering with a financial client that’s undertaking a large-scale data transformation, modernising legacy systems and building a unified analytics platform in the cloud. As part of this initiative, you’ll play a key role in developing and optimising a … Snowflake-based data warehouse and delivering high-quality visualisations and reporting through Power BI. The role offers the chance to work on greenfield architecture, migrate legacy SQL Server solutions and collaborate closely with senior stakeholders on complex, high-impact data initiatives. What You’ll Be Doing Design, develop and maintain data pipelines and models … using Snowflake, dbt and Python. Enhance and support Power BI dashboards and data models, ensuring performance, accuracy and user accessibility. Contribute to the migration of legacy SQL Server (SSIS/SSAS/SSRS) systems into a modern Snowflake/Power BI environment. Work collaboratively with business users to understand requirements, translate them into technical solutions and deliver robust More ❯

around £500/day Starting ASAP for 6 months - likely to extend About the Role We’re partnering with a financial client that’s undertaking a large-scale data transformation, modernising legacy systems and building a unified analytics platform in the cloud. As part of this initiative, you’ll play a key role in developing and optimising a … Snowflake-based data warehouse and delivering high-quality visualisations and reporting through Power BI. The role offers the chance to work on greenfield architecture, migrate legacy SQL Server solutions and collaborate closely with senior stakeholders on complex, high-impact data initiatives. What You’ll Be Doing Design, develop and maintain data pipelines and models … using Snowflake, dbt and Python. Enhance and support Power BI dashboards and data models, ensuring performance, accuracy and user accessibility. Contribute to the migration of legacy SQL Server (SSIS/SSAS/SSRS) systems into a modern Snowflake/Power BI environment. Work collaboratively with business users to understand requirements, translate them into technical solutions and deliver robust More ❯

around £500/day Starting ASAP for 6 months - likely to extend About the Role We’re partnering with a financial client that’s undertaking a large-scale data transformation, modernising legacy systems and building a unified analytics platform in the cloud. As part of this initiative, you’ll play a key role in developing and optimising a … Snowflake-based data warehouse and delivering high-quality visualisations and reporting through Power BI. The role offers the chance to work on greenfield architecture, migrate legacy SQL Server solutions and collaborate closely with senior stakeholders on complex, high-impact data initiatives. What You’ll Be Doing Design, develop and maintain data pipelines and models … using Snowflake, dbt and Python. Enhance and support Power BI dashboards and data models, ensuring performance, accuracy and user accessibility. Contribute to the migration of legacy SQL Server (SSIS/SSAS/SSRS) systems into a modern Snowflake/Power BI environment. Work collaboratively with business users to understand requirements, translate them into technical solutions and deliver robust More ❯

MI & Data Integration Analyst (Azure/Databricks) Hybrid - Central London | £43,000 pro rata | 3-month FTC (potential to extend) We're looking for a technically strong and commercially aware MI & Data Integration Analyst to join a leading UK retail and technology business on an initial 3-month fixed-term contract . This is a great … opportunity for someone who enjoys combining hands-on data analysis, reporting, and Azure integration work - helping build robust data foundations and self-serve reporting capability for an evolving business area. The Opportunity You'll be supporting a small, forward-thinking team to improve how they use data - helping them access accurate and timely MI … streamline processes, and migrate to a new Azure-based data environment. You'll take ownership of reporting improvements, work closely with third-party partners on offshoring data processes, and support the move of key datasets into the Unity Catalog environment. Key Responsibilities Produce and enhance accurate, timely, and insightful MI reports to support business decision-making. More ❯

Mid-Level Data Analyst - Outside IR35 Contract - SQL, Python, ETL, Data Pipelines I've had a new Mid-Level Data Analyst contract role come live for an InsureTech Client of OB's, who are looking for a highly skilled Mid-Level Data Analyst to join the team on a 6-month contract. … You'll be working closely with the Data Architect, setting up a Data & AI roadmap and helping to analyse how they use Data that comes through their platform. Mid-Level Data Analyst - Outside IR35 Contract - SQL, Python, ETL, Data Pipelines Key skills and experience: 3+ years of experience as a … Data Analyst SQL, Python, ETL, Data Pipelines Pays £450-£500 Outside IR35 6-month initial engagement 2-3 days a week in Central London offices Mid-Level Data Analyst - Outside IR35 Contract - SQL, Python, ETL, Data Pipelines More ❯

Mid-Level Data Analyst - Outside IR35 Contract - SQL, Python, ETL, Data Pipelines I've had a new Mid-Level Data Analyst contract role come live for an InsureTech Client of OB's, who are looking for a highly skilled Mid-Level Data Analyst to join the team on a 6-month contract. … You'll be working closely with the Data Architect, setting up a Data & AI roadmap and helping to analyse how they use Data that comes through their platform. Mid-Level Data Analyst - Outside IR35 Contract - SQL, Python, ETL, Data Pipelines Key skills and experience: 3+ years of experience as a … Data Analyst SQL, Python, ETL, Data Pipelines Pays £450-£500 Outside IR35 6-month initial engagement 2-3 days a week in Central London offices Mid-Level Data Analyst - Outside IR35 Contract - SQL, Python, ETL, Data Pipelines More ❯

Data Engineering Lead Salary: Up to £95,000 + Bonus I am working with a technology driven organisation who are leaders in their field that are delivering some of the world's most innovative projects in their space. With a strong focus on innovation, sustainability and data-driven decision-making, they are investing heavily in their … digital transformation journey. As part of this growth, they are looking to bring on a Data Engineering Lead to shape and optimise their hybrid Azure/Databricks data platform. This is a strategic and hands-on role where you will lead the development of scalable data solutions and mentor a team of BI Developers … a collaborative environment that values technical excellence, continuous learning and inclusive leadership. In this role, you will be responsible for: Leading a technical team to deliver dashboards and data products aligned with business needs. Designing and developing robust data pipelines using Azure Data Factory, Databricks, and SQL. Architecting a scalable, high-performance data More ❯

Data Engineering Lead Salary: Up to £95,000 + Bonus I am working with a technology driven organisation who are leaders in their field that are delivering some of the world's most innovative projects in their space. With a strong focus on innovation, sustainability and data-driven decision-making, they are investing heavily in their … digital transformation journey. As part of this growth, they are looking to bring on a Data Engineering Lead to shape and optimise their hybrid Azure/Databricks data platform. This is a strategic and hands-on role where you will lead the development of scalable data solutions and mentor a team of BI Developers … a collaborative environment that values technical excellence, continuous learning and inclusive leadership. In this role, you will be responsible for: Leading a technical team to deliver dashboards and data products aligned with business needs. Designing and developing robust data pipelines using Azure Data Factory, Databricks, and SQL. Architecting a scalable, high-performance data More ❯

Data Engineer - GCP £750 - £800 per day 2 days in London We're working with a global healthcare and life sciences leader that's pioneering the use of AI and Machine Learning to develop advanced therapies for both existing and emerging diseases. Their mission is to make personalised treatments faster, more effective, and more accessible - and this role … is key to that vision. The team is building cutting-edge data infrastructure to power scientific and ML applications, and they're seeking a Data Engineer with strong experience developing scalable, high-quality data pipelines in the cloud. The Role You'll join a team of data scientists, bioinformaticians, and engineers working … at the intersection of healthcare and AI. Your focus will be on designing and maintaining the data pipelines that feed large-scale ML and research workflows. Day-to-day responsibilities include: Building and maintaining data pipelines using Python, SQL, Spark, and Google Cloud technologies (BigQuery, Cloud Storage). Ensuring pipelines are robust, reliable, and optimised for More ❯

HMND 01 is a next-gen labour automation unit, providing highly efficient services across various use cases, starting with industrial applications. We are looking for Senior/Staff Data Engineer to join our team based in London, UK. Our Mission At Humanoid we strive to create the world’s leading, commercially scalable, safe, and advanced humanoid robots that … needs. Responsibilities Curate, preprocess, and manage large datasets used for training humanoid robots — projected to reach hundreds of terabytes by 2025. Ensure the quality, accuracy, and consistency of data across multiple projects. Collaborate with machine learning teams to design data pipelines that support efficient training workflows. Develop and maintain data quality metrics reporting systems. … Implement best practices for data management, including versioning, security, and compliance. Expertise Bachelor’s or Master’s degree in Data Science, Computer Science, or a related field. 5+ years of experience in data engineering and data quality. Strong proficiency in Python/Java, SQL, and data processing frameworks including Apache More ❯

HMND 01 is a next-gen labour automation unit, providing highly efficient services across various use cases, starting with industrial applications. We are looking for Senior/Staff Data Engineer to join our team based in London, UK. Our Mission At Humanoid we strive to create the world’s leading, commercially scalable, safe, and advanced humanoid robots that … needs. Responsibilities Curate, preprocess, and manage large datasets used for training humanoid robots — projected to reach hundreds of terabytes by 2025. Ensure the quality, accuracy, and consistency of data across multiple projects. Collaborate with machine learning teams to design data pipelines that support efficient training workflows. Develop and maintain data quality metrics reporting systems. … Implement best practices for data management, including versioning, security, and compliance. Expertise Bachelor’s or Master’s degree in Data Science, Computer Science, or a related field. 5+ years of experience in data engineering and data quality. Strong proficiency in Python/Java, SQL, and data processing frameworks including Apache More ❯

HMND 01 is a next-gen labour automation unit, providing highly efficient services across various use cases, starting with industrial applications. We are looking for Senior/Staff Data Engineer to join our team based in London, UK. Our Mission At Humanoid we strive to create the world’s leading, commercially scalable, safe, and advanced humanoid robots that … needs. Responsibilities Curate, preprocess, and manage large datasets used for training humanoid robots — projected to reach hundreds of terabytes by 2025. Ensure the quality, accuracy, and consistency of data across multiple projects. Collaborate with machine learning teams to design data pipelines that support efficient training workflows. Develop and maintain data quality metrics reporting systems. … Implement best practices for data management, including versioning, security, and compliance. Expertise Bachelor’s or Master’s degree in Data Science, Computer Science, or a related field. 5+ years of experience in data engineering and data quality. Strong proficiency in Python/Java, SQL, and data processing frameworks including Apache More ❯