276 to 300 of 755 Data Pipeline Jobs in London

Possibility of remote work: Hybrid (number of WFO days will depend on the client's needs) Contract duration: 6 months Location: London JOB DETAILS Role Title: Senior Data Engineer Note: (Please do not submit the same profiles as for 111721-1) Required Core Skills: Databricks, AWS, Python, Pyspark, data modelling Minimum years of experience: 7 years … Job Description: Must have hands-on experience in designing, developing, and maintaining data pipelines and data streams. Must have a strong working knowledge of moving/transforming data across layers (Bronze, Silver, Gold) using ADF, Python, and PySpark. Must have hands-on experience with PySpark, Python, AWS, data modelling. Must have experience … in ETL processes. Must have hands-on experience in Databricks development. Good to have experience in developing and maintaining data integrity and accuracy, data governance, and data security policies and procedures. Must have hands-on experience in SQL development. Excellent communication skills & logical thinking. More ❯

DATA ENGINEER - INVESTMENT FIRM 6-MONTH CONTRACT £600 PER DAY INSIDE IR35 HYBRID ABOUT THE COMPANY A rapidly growing investment firm headquartered in London, which has doubled its team over the past 2.5 years and continues to attract strong investor backing. With 230+ employees and ongoing expansion, the company is now focused on becoming more AI- and data … THE TEAM You'll join a skilled and collaborative analytics team of eight, working closely with senior leadership. The team has already laid the foundations for a modern data platform using Azure and Databricks and is now focused on building out scalable ETL processes, integrating AI tools, and delivering bespoke analytics solutions across the organisation. THE ROLE As … a Data Engineer, you'll play a pivotal role in designing and implementing robust data pipelines, supporting the migration from legacy Azure systems to Databricks, and working closely with stakeholders to deliver tailored data solutions. This role combines hands-on development with collaborative architecture design, and offers the opportunity to contribute to AI readiness More ❯

Data Engineer - Lloyd's Broker - Up to £60,000 A leading Lloyd's broker is seeking a Data Engineer to join their dynamic team. This is a key role within a growing organisation, offering the opportunity to contribute to critical data initiatives that support strategic decision-making and advanced analytics. Key Responsibilities: Design, build … and maintain robust data pipelines and datasets Ensure high data quality and integrity across systems Enable data-driven processes across business intelligence and analytics platforms Key Requirements: Proven experience as a Data Engineer within insurance (essential) Strong proficiency in SQL and Python Hands-on experience with ETL frameworks and modern data tools Familiarity with cloud platforms, ideally Microsoft Azure Solid understanding of data modelling, data warehousing, and data architecture principles This is an excellent opportunity to join a respected Lloyd's broker and play a pivotal role in shaping its data landscape. To express interest or learn more, please contact me More ❯

Senior Data Engineer – Data Modelling & Engineering – London Area – 6 Months Contract We are seeking a highly skilled and collaborative Senior Data Engineer to join a dynamic Data Modelling and Engineering team. You will play a key role in building and maintaining data pipelines, enabling seamless access to trusted data for analysts, performance teams, and data scientists. Your work will directly support the development of robust, automated, and maintainable data processes that drive confident, data-informed decision-making.This is an exciting opportunity to join a forward-thinking organisation focused on modernising and stabilising complex data landscapes. You’ll be part … and user-centric design, working in a supportive environment that encourages continuous learning and improvement. About the Role: You will work closely with upstream teams to bring critical data onto an Analytical Platform and help develop a new data layer. Your responsibilities will include establishing connectivity with downstream systems, conducting user needs analysis, and contributing to More ❯

an ever-changing global market. About the Role: At CRU, we're transforming how the world understands commodities. We're building cutting-edge platforms that will redefine how data is delivered to our clients-and we're looking for a talented Data Engineer to be at the heart of it. If you're passionate about data … and creating robust, scalable pipelines using modern tech, this is your opportunity to help shape a key part of CRU's future. Key Responsibilities: Develop and maintain scalable data pipelines to power next-gen data products in the commodities industry. Ensure data quality using the latest analytics and monitoring tools. Design and build robust … . 2-3 years of hands-on Python (scripting and development). Experience in a fast-paced startup or agile environment. Strong background in schema design and dimensional data modeling. Able to communicate data architecture clearly with internal stakeholders. Experience with Azure, Airflow, DBT, Kubernetes, GitHub. Bonus points for: open-source contributions, an active GitHub profile More ❯

an ever-changing global market. About the Role: At CRU, we're transforming how the world understands commodities. We're building cutting-edge platforms that will redefine how data is delivered to our clients-and we're looking for a talented Data Engineer to be at the heart of it. If you're passionate about data … and creating robust, scalable pipelines using modern tech, this is your opportunity to help shape a key part of CRU's future. Key Responsibilities: Develop and maintain scalable data pipelines to power next-gen data products in the commodities industry. Ensure data quality using the latest analytics and monitoring tools. Design and build robust … . 2-3 years of hands-on Python (scripting and development). Experience in a fast-paced startup or agile environment. Strong background in schema design and dimensional data modeling. Able to communicate data architecture clearly with internal stakeholders. Experience with Azure, Airflow, DBT, Kubernetes, GitHub. Bonus points for: open-source contributions, an active GitHub profile More ❯

Principal Data Engineer – AWS Location: London (Hybrid/Flexible) - Other UK Locations available Salary: £95,000 - £105,000 + Bonus A leading consultancy is hiring for a Principal Data Engineer to join their fast-growing data team, delivering cloud-native data platforms for high-profile clients across sectors. This role will focus … on designing and implementing end-to-end data solutions using modern AWS tooling. It suits someone who thrives in greenfield environments, enjoys client engagement, and values clean, scalable, well-documented engineering. Key Responsibilities: Design and build robust data pipelines using AWS (S3, Redshift, Glue, Lambda, Step Functions, DynamoDB). Deliver ETL/ELT solutions with Matillion … engineers. Requirements: Strong hands-on AWS experience – S3, Redshift, Glue essential. Proven experience building ETL/ELT pipelines in cloud environments. Proficient in working with structured/unstructured data (JSON, XML, CSV, Parquet). Skilled in working with relational databases and data lake architectures. Experienced with Matillion and modern data visualisation tools (QuickSight, Tableau More ❯

Principal Data Engineer – AWS Location: London (Hybrid/Flexible) - Other UK Locations available Salary: £95,000 - £105,000 + Bonus A leading consultancy is hiring for a Principal Data Engineer to join their fast-growing data team, delivering cloud-native data platforms for high-profile clients across sectors. This role will focus … on designing and implementing end-to-end data solutions using modern AWS tooling. It suits someone who thrives in greenfield environments, enjoys client engagement, and values clean, scalable, well-documented engineering. Key Responsibilities: Design and build robust data pipelines using AWS (S3, Redshift, Glue, Lambda, Step Functions, DynamoDB). Deliver ETL/ELT solutions with Matillion … engineers. Requirements: Strong hands-on AWS experience – S3, Redshift, Glue essential. Proven experience building ETL/ELT pipelines in cloud environments. Proficient in working with structured/unstructured data (JSON, XML, CSV, Parquet). Skilled in working with relational databases and data lake architectures. Experienced with Matillion and modern data visualisation tools (QuickSight, Tableau More ❯

Lead Data Engineer | London hybrid | Insurance The Opportunity Are you passionate about building cutting-edge data platforms that drive business growth? Our client who is a prominent London Market insurer is looking for a skilled and motivated Lead Data Engineer to play a key role in the creation of a brand-new data platform within the Azure ecosystem and Databricks. This is an exciting opportunity to be at the forefront of data innovation, working within a newly formed Data & Analytics team in a long standing London Market Insurer. You’ll help shape the data strategy, improve data quality, and empower the business to … make data-driven decisions. As a Lead Data Engineer, you'll work closely with both technical and business stakeholders, leveraging your expertise to design, develop, and optimize a high-performance data platform built on Databricks. This platform will be built to scale, incorporating the latest advancements in data intelligence while supporting strategic More ❯

Lead Data Engineer | London hybrid | Insurance The Opportunity Are you passionate about building cutting-edge data platforms that drive business growth? Our client who is a prominent London Market insurer is looking for a skilled and motivated Lead Data Engineer to play a key role in the creation of a brand-new data platform within the Azure ecosystem and Databricks. This is an exciting opportunity to be at the forefront of data innovation, working within a newly formed Data & Analytics team in a long standing London Market Insurer. You’ll help shape the data strategy, improve data quality, and empower the business to … make data-driven decisions. As a Lead Data Engineer, you'll work closely with both technical and business stakeholders, leveraging your expertise to design, develop, and optimize a high-performance data platform built on Databricks. This platform will be built to scale, incorporating the latest advancements in data intelligence while supporting strategic More ❯

Principal Data Engineer £95,000 - £105,000 + BonusPermanent - London/Bristol or Leeds (Hybrid) A top-tier global consultancy is in search of a Data Engineer (Principal Consultant) to drive the delivery of cloud-native data platforms across complex, high-impact environments. This is a standout opportunity to join a fast-growing consulting … team where you’ll lead technically, shape strategy, and support business development efforts across public and private sector clients. If you’re passionate about data engineering, experienced in building practices and winning new work (RFI/RFP), and want to work in an organisation that values creativity, autonomy, and technical excellence — this could be the one. What you … ll do as a Principal Data Engineer: Lead the architecture and build of modern data platforms using AWS-native tools (S3, Redshift, Lambda, Glue, DynamoDB, Matillion, etc.). Drive best practices in CI/CD, Infrastructure-as-Code, and data ops methodologies. Liaise directly with clients to define requirements, shape technical solutions, and oversee More ❯

Data Engineer – Python | Databricks | PySpark Company: Fortune 500 Financial Services firm Location: Hybrid - London Type: Permanent Salary: £90k + 20% bonus + Exceptional Benefits Exclusively via DATAHEAD Are you a detail-oriented Python Developer who thrives in complex data environments? Do you have hands-on experience with Databricks , PySpark , and cloud-native data engineering … We’re hiring a Data Engineer on behalf of a Fortune 500 firm undergoing a major data and analytics transformation. You’ll join a global Advanced Analytics function working at the cutting edge of data science, engineering, and AI. Why this role? You’ll join a cross-functional team that includes data scientists, actuaries, software engineers, and AI specialists - all working together to build intelligent data platforms that power business-critical decisions across the organisation. This is a cloud-first, engineering-led environment where you’ll play a key role in developing high-quality, scalable data products. What You’ll Do: Build and maintain scalable Python More ❯

Data Engineer – Python | Databricks | PySpark Company: Fortune 500 Financial Services firm Location: Hybrid - London Type: Permanent Salary: £90k + 20% bonus + Exceptional Benefits Exclusively via DATAHEAD Are you a detail-oriented Python Developer who thrives in complex data environments? Do you have hands-on experience with Databricks , PySpark , and cloud-native data engineering … We’re hiring a Data Engineer on behalf of a Fortune 500 firm undergoing a major data and analytics transformation. You’ll join a global Advanced Analytics function working at the cutting edge of data science, engineering, and AI. Why this role? You’ll join a cross-functional team that includes data scientists, actuaries, software engineers, and AI specialists - all working together to build intelligent data platforms that power business-critical decisions across the organisation. This is a cloud-first, engineering-led environment where you’ll play a key role in developing high-quality, scalable data products. What You’ll Do: Build and maintain scalable Python More ❯

Junior AI Data Engineer Department: Software Engineering Employment Type: Full Time Location: London Description Zencargo is looking for an AI Data Engineer to join our collaborative and forward thinking software team. In this role, you'll play a key part in shaping how we use data to power intelligent agents and generative AI features … bringing structure, stability, and context to real-world supply chain problems. You bridge the gap between data systems and intelligent agents, enabling the Zencargo team to move from clever prompts to stable, contextualised AI features. We're looking for someone who understands the data context for AI, feeds models intelligently, and brings structure and reliability to … mentorship and opportunities to develop your skills in a supportive environment. Key Responsibilities Over the next 12 months, we would expect you to become proficient in the following: Data Solutions and AI context engineering Translate complex business processes and workflows into structured, logical input that agents and models can understand. Design, build, and deploy Python solutions that enhance More ❯

Backend Data Engineer Department: Development Team Employment Type: Full Time Location: London Description We are seeking a talented Backend Data Engineer to join our Core Technology Team, where you'll specialize in Python development to create robust automation solutions for both trading and non-trading back-office operations. This role sits at the intersection of technology … our trading platform and supporting infrastructure. As a key member of our Core Technology Team, you'll be responsible for streamlining complex workflows, automating manual processes, and delivering data-driven insights that enable our trading operations to scale effectively. You'll work closely with traders, operations staff, and other technology teams to identify automation opportunities and implement solutions … reduce operational overhead while maintaining the reliability and precision essential to our business. Key Responsibilities Reporting systems: building both real-time and scheduled reporting tools for various stakeholders Data integration: connecting and consolidating data from multiple sources Tooling development and maintenance : building new automation tools and ensuring their reliability and continuous improvement Collaborative problem-solving: working More ❯

Data Engineer London/Hybrid - Europe/Remote About Snowplow: Snowplow is the global leader in customer data infrastructure for AI, enabling every organization to transform raw behavioral data into governed, high-fidelity fuel for AI-powered applications-including advanced analytics, real-time personalization engines, and AI agents. Digital-first companies like Strava, HelloFresh … Auto Trader, Burberry, and DPG Media use Snowplow to collect and process event-level data in real time, delivering it securely to their warehouse, lake, or stream, and integrate deep customer context into their applications. Thousands of companies rely on Snowplow to uncover customer insights, predict customer behaviors, hyper-personalize customer experiences, and detect fraud in real time. … Snowplow Signals - our new solution for enabling real-time, AI-powered customer experiences. Your work will contribute to the infrastructure that powers these capabilities, unlocking value from behavioral data in real time . Alongside this, you'll work on the foundational components that continue to make Snowplow powerful and flexible for data teams: developing and maintaining More ❯

Job Title: Data Engineer Salary: ~£75k (+ 20% bonus) Location: Covent Garden (3 office days/week) Industry: Telecoms (challenger) This MVNO company is on a mission to disrupt the mobile network industry with highly innovative ideas and a revolutionary business model that truly places customers at the heart of everything they do, rewarding them with loyalty-based … This fast-growth start-up utilises the very latest technology, setting out to be a hub of creativity and pioneering ideas that will transform the industry. As a Data Engineer, you will be an integral part of a data-driven organisation, responsible for the data infrastructure from data collection through to data service of analytics, you will provide self-servicing tools and access of data to support colleagues to make data-driven decision and drive results. This role would be perfect for someone with a strong foundation in Data Engineering (e.g. 2-3 years' experience), looking to take their career to the next level More ❯

Job Title: Data Engineer Salary: ~£75k (+ 20% bonus) Location: Covent Garden (3 office days/week) Industry: Telecoms (challenger) This MVNO company is on a mission to disrupt the mobile network industry with highly innovative ideas and a revolutionary business model that truly places customers at the heart of everything they do, rewarding them with loyalty-based … This fast-growth start-up utilises the very latest technology, setting out to be a hub of creativity and pioneering ideas that will transform the industry. As a Data Engineer, you will be an integral part of a data-driven organisation, responsible for the data infrastructure from data collection through to data service of analytics, you will provide self-servicing tools and access of data to support colleagues to make data-driven decision and drive results. This role would be perfect for someone with a strong foundation in Data Engineering (e.g. 2-3 years' experience), looking to take their career to the next level More ❯

Senior Data Engineer with 5+ years’ experience, and 2+ years in a Hedge Fund and/or in Systematic Trading Technology, sought to join a market-leading Quantitative Hedge Fund. You will join a 5-strong Data Engineering team covering the ingestion, storage, transformation and distribution of tick, timeseries, reference and alternative datasets. The technology stack … a range of legacy and modern systems, across on-premises and cloud infrastructure with technologies and tooling such as Python, dbt, KDB+, Snowflake, SQL and interfacing with Market Data vendors such as Bloomberg, Refinitiv, Factset and MorningStar. This is an exciting time for you to join the team they consolidate our technology estate, revamp how they process and … filter data, and overhaul the way data is accessed by their consumers, while continuing to onboard new datasets that enhance their strategies. They are a lean team owning end-to-end delivery from initial design through to operational support in production. The firm work on a hybrid working schedule, with a minimum of three-days-per More ❯

Senior Data Engineer with 5+ years’ experience, and 2+ years in a Hedge Fund and/or in Systematic Trading Technology, sought to join a market-leading Quantitative Hedge Fund. You will join a 5-strong Data Engineering team covering the ingestion, storage, transformation and distribution of tick, timeseries, reference and alternative datasets. The technology stack … a range of legacy and modern systems, across on-premises and cloud infrastructure with technologies and tooling such as Python, dbt, KDB+, Snowflake, SQL and interfacing with Market Data vendors such as Bloomberg, Refinitiv, Factset and MorningStar. This is an exciting time for you to join the team they consolidate our technology estate, revamp how they process and … filter data, and overhaul the way data is accessed by their consumers, while continuing to onboard new datasets that enhance their strategies. They are a lean team owning end-to-end delivery from initial design through to operational support in production. The firm work on a hybrid working schedule, with a minimum of three-days-per More ❯

boundaries of what's possible. If you are ready to be part of a transformative journey, come and join us. We are currently seeking a talented and driven Data Engineer to join our growing Digital team, playing a critical role in powering analytics and data infrastructure to support rail operations and maintenance use cases. This role … will be instrumental in shaping the future of our AWS-based data and reporting platform for Siemens Mobility UK Rolling Stock, supporting the delivery of train service contracts for our customers. You'll create impact by . Turning complex data and business requirements into scalable AWS-based data solutions, working closely with Data Scientists, Analysts, and stakeholders. Designing, building, and supporting end-to-end data pipelines and infrastructure that power analytics and reporting for Siemens Mobility UK. Leading the technical delivery of secure, reliable DevOps practices while proactively resolving emerging issues across reporting and database systems. Ensuring all AWS-native data services are fully documented, maintained, and More ❯

Data Scientist Hybrid – London (3 days a week) Up to £75,000 + Pension + Share Scheme About the Role We’re working with a professional services firm undergoing a major data transformation, now looking to bring on board their first in-house Data Scientist . Reporting directly into the CFO, you’ll take … the lead on developing and managing the firm’s data infrastructure, starting with the continued build-out and integration of their data lake. You’ll work on high-impact reporting and analytics projects, and long-term, deliver commercially focused data science solutions such as price optimisation and revenue forecasting. This is a hands-on … role in a people-first business, ideal for someone eager to take ownership, collaborate cross-functionally, and uncover opportunities to use data more effectively across the organisation. Key Responsibilities Build and manage data pipelines and infrastructure, including a developing data lake Support reporting projects and ongoing Power BI development Deliver commercial data More ❯

Data Scientist Hybrid – London (3 days a week) Up to £75,000 + Pension + Share Scheme About the Role We’re working with a professional services firm undergoing a major data transformation, now looking to bring on board their first in-house Data Scientist . Reporting directly into the CFO, you’ll take … the lead on developing and managing the firm’s data infrastructure, starting with the continued build-out and integration of their data lake. You’ll work on high-impact reporting and analytics projects, and long-term, deliver commercially focused data science solutions such as price optimisation and revenue forecasting. This is a hands-on … role in a people-first business, ideal for someone eager to take ownership, collaborate cross-functionally, and uncover opportunities to use data more effectively across the organisation. Key Responsibilities Build and manage data pipelines and infrastructure, including a developing data lake Support reporting projects and ongoing Power BI development Deliver commercial data More ❯

Data Engineer Cognify are partnering with Freetrade to help them expand their Data Team. We are searching for a Data Engineer to join them on their journey! Freetrade’s mission is to become the default place to invest. Investing has been too complicated and expensive for too long, keeping millions from making the most … mutual funds, bonds, and family investment tools. We’re also exploring how we can leverage AI to deliver an even better experience for our customers. Why join the Data team at Freetrade? Data is at the core of how we make decisions, build products, and drive the business forward. This year, we’re scaling our data platform, refining self service analytics, and partnering across the business to tackle high impact projects. If you want to work on meaningful challenges where data turns into action, this role is for you. Key Responsibilities Build the pipelines that put Freetrade’s data to work, deploying jobs into our Kubernetes cluster, writing Python code More ❯

Data Engineer Cognify are partnering with Freetrade to help them expand their Data Team. We are searching for a Data Engineer to join them on their journey! Freetrade’s mission is to become the default place to invest. Investing has been too complicated and expensive for too long, keeping millions from making the most … mutual funds, bonds, and family investment tools. We’re also exploring how we can leverage AI to deliver an even better experience for our customers. Why join the Data team at Freetrade? Data is at the core of how we make decisions, build products, and drive the business forward. This year, we’re scaling our data platform, refining self service analytics, and partnering across the business to tackle high impact projects. If you want to work on meaningful challenges where data turns into action, this role is for you. Key Responsibilities Build the pipelines that put Freetrade’s data to work, deploying jobs into our Kubernetes cluster, writing Python code More ❯