1 to 25 of 363 Data Pipeline Jobs in Central London

a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data … technical solutions. Communicate technical concepts effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote and implement DevOps best practices. … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation: Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data … technical solutions. Communicate technical concepts effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote and implement DevOps best practices. … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

drive the accelerated digitalization of our talent. As the D&A Technical Lead for Smart Factories (Global), you will play a pivotal role in shaping and executing the Data Strategy for Smart Factory within Pet Nutrition Division. You will collaborate with cross-functional and cross-regional teams to develop, implement, and optimize Smart Factories strategies that align with … promotional opportunity with Mars before December 31, 2026. What are we looking for? 10+ years’ experience working in this space, with particular focus on working with Manufacturing, CPG data At least 5 years’ experience in a senior data/architecture/leadership role with demonstrable thought leadership abilities Experience in Architecture as a code approach. Experience … such as: Azure Stack \ Data Bricks \ Python (inc. Django framework) Manufacturing data platform tools like Thingworx, Litmus is a plus Experience in data pipeline architecture. Knowledge of containerisation/microservices (Kubernetes/docker). Knowledge of API architecture REST, Webhooks, SOAP. Experience working in a multi-business unit or multi-client environment that More ❯

drive the accelerated digitalization of our talent. As the D&A Technical Lead for Smart Factories (Global), you will play a pivotal role in shaping and executing the Data Strategy for Smart Factory within Pet Nutrition Division. You will collaborate with cross-functional and cross-regional teams to develop, implement, and optimize Smart Factories strategies that align with … opportunity with Mars before December 31, 2026 . What are we looking for? 10+ years’ experience working in this space, with particular focus on working with Manufacturing, CPG data At least 5 years’ experience in a senior data/architecture/leadership role with demonstrable thought leadership abilities Experience in Architecture as a code approach. Experience … such as: Azure Stack \ Data Bricks \ Python (inc. Django framework) Manufacturing data platform tools like Thingworx, Litmus is a plus Experience in data pipeline architecture. Knowledge of containerisation/microservices (Kubernetes/docker). Knowledge of API architecture REST, Webhooks, SOAP. Experience working in a multi-business unit or multi-client environment that More ❯

IR35) Databricks Engineer needed with active SC Security Clearance for 6 Month Contract based in Central London (Hybrid). Developing a cutting-edge Azure Databricks platform for economic data modelling, analysis, and forecasting. Start ASAP in Nov/Dec 2025. Hybrid Working - 2 days/week remote (WFH), and 3 days/week working on-site from the … Central London office. A chance to work with a leading global IT and Digital transformation business specialising in Government projects: In-depth Data Engineering + strong hands-on Azure Databricks expertise. Azure Data Services, Azure Data Factory, Azure Blob Storage + Azure SQL Database. Designing, developing, building + optimising data pipelines, implementing … including data modelling techniques + data integration patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data More ❯

Engineer Location: London (hybrid 3 times a week) Department: IT Type: 3 months rolling contract Outside IR35 The client is seeking an experienced Analytics Engineer to join its Data & Analytics team. The role focuses on building scalable data pipelines, transforming raw data into clean, reusable datasets, and enabling data-driven decision-making. … Key Responsibilities Design and build data products, with proficiency throughout the data lifecycle. Develop robust data models through close collaboration with business users and the engineering team. Partner with senior management, operational leads, and other stakeholders, coaching and supporting a data-driven culture, including KPI definition and reporting frameworks. Accountable for data extraction, transforming JSON & XML, utlising high experience within metadata management. Collaborate with data engineers to develop data, enrich product design and integrate data for predictive models or machine learning. Deliver well-defined, transformed, tested, documented, and code-reviewed datasets for analysis. Evaluate and recommend improvements in data flow, influencing and More ❯

Location - London, Bristol or Manchester (1 day a month onsite) Duration - 6 months Rate - £550 - £600pd (inside ir35) As a Data Engineer in the Cyber and Domains Protection Team you will: Work within an Agile team to support the development of dashboards and build automated reports to meet the needs of technical and non-technical users Work with … the data analyst and user researcher to update relevant data models to allow business intelligence data to meet the organisation's specific needs Develop business intelligence reports that can be automated, reused and shared with users directly Implement data flows to connect operational systems, data for analytics and business intelligence … for ETL and data cataloging Amazon Redshift or Athena for data warehousing and analytics Lambda for event-driven data processing. ETL/ELT pipeline development : experience in designing, building, and maintaining robust, automated data pipelines. You should be comfortable with both the theory and practical application of extracting, transforming, and loading More ❯

Data Architect - Contract We are seeking a skilled Data Architect to lead the design, development, and oversight of our enterprise-wide data infrastructure. The Data Architect will involve data strategy and ensuring high standards of data quality, accessibility, and security across multiple systems and platforms. The ideal candidate … will be a strategic thinker with strong expertise in modern data architectures and data governance practices. Required Skills and Experience: Proven experience as a Data Architect or in a similar senior data-focused role. Extensive experience with various data modelling techniques and tools. Deep understanding of different data … solutions (e.g., relational, NoSQL, data lakes) and their appropriate use cases. Strong knowledge of data integration patterns, ETL/ELT processes, and data pipeline orchestration. Experience with AWS data platforms and their respective data services. Solid understanding of data governance principles, including data quality, metadata More ❯

Data Architect - Contract We are seeking a skilled Data Architect to lead the design, development, and oversight of our enterprise-wide data infrastructure. The Data Architect will involve data strategy and ensuring high standards of data quality, accessibility, and security across multiple systems and platforms. The ideal candidate … will be a strategic thinker with strong expertise in modern data architectures and data governance practices. Required Skills and Experience: Proven experience as a Data Architect or in a similar senior data-focused role. Extensive experience with various data modelling techniques and tools. Deep understanding of different data … solutions (e.g., relational, NoSQL, data lakes) and their appropriate use cases. Strong knowledge of data integration patterns, ETL/ELT processes, and data pipeline orchestration. Experience with AWS data platforms and their respective data services. Solid understanding of data governance principles, including data quality, metadata More ❯

Rate : £600 to £650 per day outside IR35 Duration: 6 months Location: one day a week on site in London Security Clearance: Active SC The Data Architect will have the following responsibilities: Collaborate with business and technology stakeholders to translate business problems into scalable data architecture solutions. Design, document, and maintain enterprise and solution-level data architectures across multiple platforms and domains. Define and enforce data standards, principles, and governance frameworks to ensure consistency and quality. Develop conceptual, logical, and physical data models aligned with business needs and organisational strategy. Select appropriate data storage, integration, and processing technologies for each project’s context. Evaluate emerging tools, technologies, and … understanding of data modelling (conceptual, logical, and physical), including relational, dimensional, and NoSQL approaches. Expertise in data integration, ETL/ELT, and data pipeline design. Hands-on experience with data lakehouse, warehouse, and streaming data architectures. Working knowledge of SQL, Python, and relevant data engineering frameworks (e.g. More ❯

Rate : £600 to £650 per day outside IR35 Duration: 6 months Location: one day a week on site in London Security Clearance: Active SC The Data Architect will have the following responsibilities: Collaborate with business and technology stakeholders to translate business problems into scalable data architecture solutions. Design, document, and maintain enterprise and solution-level data architectures across multiple platforms and domains. Define and enforce data standards, principles, and governance frameworks to ensure consistency and quality. Develop conceptual, logical, and physical data models aligned with business needs and organisational strategy. Select appropriate data storage, integration, and processing technologies for each project’s context. Evaluate emerging tools, technologies, and … understanding of data modelling (conceptual, logical, and physical), including relational, dimensional, and NoSQL approaches. Expertise in data integration, ETL/ELT, and data pipeline design. Hands-on experience with data lakehouse, warehouse, and streaming data architectures. Working knowledge of SQL, Python, and relevant data engineering frameworks (e.g. More ❯

Senior Data Engineer (Azure) 2-3 days on-site - central London up to £75k + 20% Bonus + Excellent Benefits Our client is a leading global Retail/FMCG brand undergoing an exciting period of rapid growth and transformation. With significant investment in data and technology, they are building a world-class data platform … across every area of the business — from supply chain and logistics to marketing, customer sales and in-store operations. As part of this journey, they are expanding their Data Engineering team and looking for a Senior Data Engineer (Azure) to help shape and scale the next generation of data products and pipelines. The Role … As a Senior Data Engineer, you’ll play a key role in designing, building, and optimising scalable data pipelines and solutions on Microsoft Azure. You’ll work closely with Data Architects, Analysts, and stakeholders in the business to expand the current data platform — which already powers sales and performance dashboards — into a More ❯

Senior Data Engineer (Azure) 2-3 days on-site - central London up to £75k + 20% Bonus + Excellent Benefits Our client is a leading global Retail/FMCG brand undergoing an exciting period of rapid growth and transformation. With significant investment in data and technology, they are building a world-class data platform … across every area of the business — from supply chain and logistics to marketing, customer sales and in-store operations. As part of this journey, they are expanding their Data Engineering team and looking for a Senior Data Engineer (Azure) to help shape and scale the next generation of data products and pipelines. The Role … As a Senior Data Engineer, you’ll play a key role in designing, building, and optimising scalable data pipelines and solutions on Microsoft Azure. You’ll work closely with Data Architects, Analysts, and stakeholders in the business to expand the current data platform — which already powers sales and performance dashboards — into a More ❯

Data Engineer (with Data Analytics Background) Location: City of London Employment Type: Full-time Salary: £90,000-£100,000 Sector: Fintech We're looking for a well-rounded, communicative Data Engineer with a strong background in data analytics and experience within the Fintech sector . This role is ideal for someone who … began their career as a Data Analyst and has since transitioned into a more engineering-focused position, someone who enjoys understanding the business context just as much as building the data solutions behind it. You'll work extensively with Python , Snowflake , SQL , and dbt to design, build, and maintain scalable, high-quality data pipelines … and models that support decision-making across the business. This is a hands-on, collaborative role, suited to someone who's confident communicating with data, product, and engineering teams, not a 'heads-down coder' type. Top 4 Core Skills Python - workflow automation, data processing, and ETL/ELT development. Snowflake - scalable data architecture, performance More ❯

Data Engineer (with Data Analytics Background) Location: City of London Employment Type: Full-time Salary: £90,000–£100,000 Sector: Fintech We’re looking for a well-rounded, communicative Data Engineer with a strong background in data analytics and experience within the Fintech sector . This role is ideal for someone who … began their career as a Data Analyst and has since transitioned into a more engineering-focused position, someone who enjoys understanding the business context just as much as building the data solutions behind it. You’ll work extensively with Python , Snowflake , SQL , and dbt to design, build, and maintain scalable, high-quality data pipelines … and models that support decision-making across the business. This is a hands-on, collaborative role, suited to someone who’s confident communicating with data, product, and engineering teams, not a “heads-down coder” type. Top 4 Core Skills Python — workflow automation, data processing, and ETL/ELT development. Snowflake — scalable data architecture, performance More ❯

Senior Data Engineer - Trino/Starburst 🚀 Senior Data Engineer Opportunity Salary: Up to £75k (Based on candidate experience) + Excellent benefits Work commute Model: Mostly remote with just 1 day per week on-site Location: London/Portsmouth Job Type: Permanent Ready to Shape the Future of Data? My client is a leading UK … based IT services and consulting company seeking an experienced Senior Data Engineer who thrives on building robust, scalable systems that unlock the true potential of enterprise data. This is your opportunity to join their cutting-edge AI & Data team as a Senior Data Engineer . What You'll Be Doing 🔧 Build & Optimise : Take ownership … and/or Java 🤖 Automation Expertise with tools like Ansible 🧠 AI/ML Data Requirements understanding 📊 Data Federation and caching technologies experience 🔄 Data Pipeline design and optimisation 🏢 Dell Data Lakehouse administration background Services offered by Computappoint Limited are those of an Employment Business and/or Employment Agency in relation to More ❯

Data Engineer – Python | SQL | Azure | Microsoft Fabric Location: London Salary: £90,000 + benefits Financial Services Are you a skilled Data Engineer with strong coding expertise and a passion for building scalable data solutions? We’re looking for a talented professional to join our team and play a key role in designing, developing, and … optimising modern data platforms on Azure Cloud — with a focus on an exciting Microsoft Fabric implementation. What you’ll do Design and build robust data pipelines and ETL processes using Python and SQL. Contribute to the rollout and optimisation of Microsoft Fabric, shaping the next-generation data platform. Work across Azure services (Data Factory, Synapse, Databricks, Data Lake, Functions) to deliver secure and scalable data solutions. Optimise data workflows for performance, reliability, and cost efficiency. Collaborate with analysts, data scientists, and business stakeholders to ensure high-quality, trusted data is available for decision-making. Implement best practices for data More ❯

Data Engineer – Python | SQL | Azure | Microsoft Fabric Location: London Salary: £90,000 + benefits Financial Services Are you a skilled Data Engineer with strong coding expertise and a passion for building scalable data solutions? We’re looking for a talented professional to join our team and play a key role in designing, developing, and … optimising modern data platforms on Azure Cloud — with a focus on an exciting Microsoft Fabric implementation. What you’ll do Design and build robust data pipelines and ETL processes using Python and SQL. Contribute to the rollout and optimisation of Microsoft Fabric, shaping the next-generation data platform. Work across Azure services (Data Factory, Synapse, Databricks, Data Lake, Functions) to deliver secure and scalable data solutions. Optimise data workflows for performance, reliability, and cost efficiency. Collaborate with analysts, data scientists, and business stakeholders to ensure high-quality, trusted data is available for decision-making. Implement best practices for data More ❯

Remote Data Architect sought by a well-established company in their industry; our client is seeking an adept professional to spearhead strategic leadership and technical expertise within their data solutions landscape, focussed on cloud data platforms, but including IoT analytics, data integration, and migration projects. This unique opportunity is offered on a … remote-first basis, but with an open invitation to their headquarters any time. Role Responsibilities: The Data Architect will be pivotal in shaping the approach to designing and developing data solutions using cloud technologies such as AWS and Azure. This position involves: - Leading the design and implementation of innovative cloud data platforms. - Setting standards … using Azure, AWS, or exceptional on-premise design expertise. - At least 5 years in data engineering or business intelligence roles. - Proficiency in ETL and data pipeline design, with a technology-agnostic approach. - A solid understanding of data warehouse and data lake principles. - Expert SQL skills and demonstrable data modelling More ❯

Lead Data Engineer - London - Hybrid - £85,000 - £90,000 My client is seeking a highly skilled Lead Data Engineer to join their agile data team in London. This hybrid position requires 3 days per week in the office, offering the opportunity to lead a small team while remaining technically hands-on. The role is … annual leave plus buy and sell options Hybrid working: 3 days per week in London office And many more Role and Responsibilities Architect, develop, and continuously improve robust data pipelines to support large-scale data ingestion, transformation, and delivery. Utilize Azure and Databricks platforms to ensure high-throughput and reliable data … movement across diverse data sources, enabling seamless integration, cleansing, and enrichment of data for analytics and reporting needs. Implement best practices for data pipeline orchestration, monitoring, and error handling to ensure consistent and dependable data flows. Optimize and fine-tune Azure-based data platforms by leveraging deep expertise in More ❯

Lead Data Engineer - London - Hybrid - £85,000 - £90,000 My client is seeking a highly skilled Lead Data Engineer to join their agile data team in London. This hybrid position requires 3 days per week in the office, offering the opportunity to lead a small team while remaining technically hands-on. The role is … annual leave plus buy and sell options Hybrid working: 3 days per week in London office And many more Role and Responsibilities Architect, develop, and continuously improve robust data pipelines to support large-scale data ingestion, transformation, and delivery. Utilize Azure and Databricks platforms to ensure high-throughput and reliable data … movement across diverse data sources, enabling seamless integration, cleansing, and enrichment of data for analytics and reporting needs. Implement best practices for data pipeline orchestration, monitoring, and error handling to ensure consistent and dependable data flows. Optimize and fine-tune Azure-based data platforms by leveraging deep expertise in More ❯

Data Engineer | Location: London (Hybrid - 2 days WFO) | Salary: £60,000-£80,000 | London Market Insurance experience ESSENTIAL. The Opportunity Are you passionate about building cutting-edge data platforms that drive business growth? Our client is seeking who is a Speciality Insurer a skilled and motivated Data Engineer with Insurance experience to play a … key role in the creation of a brand-new data platform within the Azure ecosystem and Databricks . This is an exciting opportunity to be at the forefront of data innovation, working within a newly formed Data & Analytics team in a long standing London Market Insurer . You’ll help shape the data strategy, improve data quality, and empower the business to make data-driven decisions. As a Lead Data Engineer, you'll work closely with both technical and business stakeholders, leveraging your expertise to design, develop, and optimize a high-performance data platform built on Databricks . This platform will be built More ❯

Data Engineer | Location: London (Hybrid - 2 days WFO) | Salary: £60,000-£80,000 | London Market Insurance experience ESSENTIAL. The Opportunity Are you passionate about building cutting-edge data platforms that drive business growth? Our client is seeking who is a Speciality Insurer a skilled and motivated Data Engineer with Insurance experience to play a … key role in the creation of a brand-new data platform within the Azure ecosystem and Databricks . This is an exciting opportunity to be at the forefront of data innovation, working within a newly formed Data & Analytics team in a long standing London Market Insurer . You’ll help shape the data strategy, improve data quality, and empower the business to make data-driven decisions. As a Lead Data Engineer, you'll work closely with both technical and business stakeholders, leveraging your expertise to design, develop, and optimize a high-performance data platform built on Databricks . This platform will be built More ❯

Data Developer (Microsoft Fabric/Azure) Location: Essex or Southampton (on-site, parking available) Salary: £45,000 - £56,000 per annum The Opportunity La Fosse is partnering with a leading global logistics organisation on an exciting data transformation project. We’re seeking an experienced Data Developer to join a new three-person data … Systems. This role will play a key part in the company’s migration from QlikView to Microsoft Fabric, helping to design and deliver the next generation of their data environment using Azure technologies. Key Responsibilities Lead and support the QlikView → Microsoft Fabric migration Build and maintain ETL processes and data pipelines Develop scalable solutions in Azure … analytics Collaborate across teams to enhance data quality and governance Key Requirements Proven Microsoft Fabric and Azure Data Factory experience Strong ETL, data pipeline, and SQL skills Expertise in Power BI and data visualisation Background as a Data Developer/Data Engineer Experience migrating from QlikView (essential More ❯

Data Developer (Microsoft Fabric/Azure) Location: Essex or Southampton (on-site, parking available) Salary: £45,000 - £56,000 per annum The Opportunity La Fosse is partnering with a leading global logistics organisation on an exciting data transformation project. We’re seeking an experienced Data Developer to join a new three-person data … Systems. This role will play a key part in the company’s migration from QlikView to Microsoft Fabric, helping to design and deliver the next generation of their data environment using Azure technologies. Key Responsibilities Lead and support the QlikView → Microsoft Fabric migration Build and maintain ETL processes and data pipelines Develop scalable solutions in Azure … analytics Collaborate across teams to enhance data quality and governance Key Requirements Proven Microsoft Fabric and Azure Data Factory experience Strong ETL, data pipeline, and SQL skills Expertise in Power BI and data visualisation Background as a Data Developer/Data Engineer Experience migrating from QlikView (essential More ❯