1 to 25 of 31 Data Pipeline Jobs in the City of London

Technical Data Specialist Contract onsite in London Day rate: £650 - £858 per day inside IR35 We're currently seeking an accomplished and highly ambitious Technical Data Specialist to work with our exceptional client, a world class brand in a secure environment to drive major business transformation … process and technical change across this complex organisation. This contract roles will require you to conduct security clearance prior to assignment. The Technical Data Specialist role is conducted in an environment that is far from ordinary, therefore, we're not looking for ordinary. The Technical Data Specialists set the vision for the organisation's use of data (through data design) to ensure that data is managed properly and meets the organisation's needs. You will work with key internal and external partners to deliver data architecture More ❯

Head of Data Engineering – Databricks, Azure, Data Strategy, Data Governance, Leadership, Management, Azure, Python, ETL, Data Pipelines, Private Markets A global Private Equity firm are seeking an experienced Head of Data Engineering as part of a critical data transformation and modernisation programme, moving towards a data-first approach. They are currently in the process of replacing a legacy data-warehouse setup with a platform as a service model using Databricks hosted on Azure, which this role will be responsible for leading, taking … line management responsibilities for their Data Engineering team. You will take ownership for progressing their Databricks setup, from its current PoC/incubator phase through to production, ensuring a high level of optimisation and scalability to operate at a global scale. This will also involve implementing a semantic More ❯

Head of Data Engineering – Databricks, Azure, Data Strategy, Data Governance, Leadership, Management, Azure, Python, ETL, Data Pipelines, Private Markets A global Private Equity firm are seeking an experienced Head of Data Engineering as part of a critical data transformation and modernisation programme, moving towards a data-first approach. They are currently in the process of replacing a legacy data-warehouse setup with a platform as a service model using Databricks hosted on Azure, which this role will be responsible for leading, taking … line management responsibilities for their Data Engineering team. You will take ownership for progressing their Databricks setup, from its current PoC/incubator phase through to production, ensuring a high level of optimisation and scalability to operate at a global scale. This will also involve implementing a semantic More ❯

Head of Data Engineering – Databricks, Azure, Data Strategy, Data Governance, Leadership, Management, Azure, Python, ETL, Data Pipelines, Private Markets A global Private Equity firm are seeking an experienced Head of Data Engineering as part of a critical data transformation and modernisation programme, moving towards a data-first approach. They are currently in the process of replacing a legacy data-warehouse setup with a platform as a service model using Databricks hosted on Azure, which this role will be responsible for leading, taking … line management responsibilities for their Data Engineering team. You will take ownership for progressing their Databricks setup, from its current PoC/incubator phase through to production, ensuring a high level of optimisation and scalability to operate at a global scale. This will also involve implementing a semantic More ❯

Data Engineer (Snowflake) - Nationwide Charity Location: London-based (Hybrid - 1 day per week in the office) Contract: FTC - 23 months £56,500 Overview: We are currently recruiting on behalf of a well-established nationwide charity for a Snowflake Data Engineer . This is an exciting opportunity … to play a key role in designing and developing data products and services that improve how data is accessed and used across the organisation. You'll help ensure data is accurate, timely, and fit for purpose, supporting a wide range of departments and strategic … goals. Role Summary: As a Data Engineer, you will work to connect systems, streamline business processes, and optimise data warehouse platforms. You'll collaborate with stakeholders to deliver bespoke BI solutions, ranging from custom queries to dashboards and reports. Your work will transform raw data More ❯

Data Engineer (Snowflake) - Nationwide Charity Location: London-based (Hybrid - 1 day per week in the office) Contract: FTC - 23 months £56,500 Overview: We are currently recruiting on behalf of a well-established nationwide charity for a Snowflake Data Engineer . This is an exciting opportunity … to play a key role in designing and developing data products and services that improve how data is accessed and used across the organisation. You'll help ensure data is accurate, timely, and fit for purpose, supporting a wide range of departments and strategic … goals. Role Summary: As a Data Engineer, you will work to connect systems, streamline business processes, and optimise data warehouse platforms. You'll collaborate with stakeholders to deliver bespoke BI solutions, ranging from custom queries to dashboards and reports. Your work will transform raw data More ❯

Data Production Engineer Location: London, United Kingdom Salary: Competitive + Excellent Benefits Our client, a leading global trading firm, is seeking a talented Data Production Engineer to join their highly collaborative Data team. Data is central to their success, powering one of … the unique opportunity to work directly with live trading teams, automate processes, explore vast datasets, and engage with key external stakeholders such as data vendors, brokers, and exchanges. You'll play a hands-on role in acquiring, validating, and preparing data that feeds cutting-edge quantitative … Comfortable using the Linux command line for file manipulation, automation, and system monitoring. Experience with financial datasets (e.g. Refinitiv, S&P, Bloomberg) and ETL pipeline management is highly desirable. Prior exposure to supporting systems in a production trading environment is a strong advantage. Why Apply? You'll join a More ❯

Data Solutions Architect Hybrid - Inside IR35 We're looking for a seasoned Data Solutions Architect with deep expertise in the Lloyd's of London Market and a strong background in Snowflake and ETL/data transformation tools . As a Data Architect … you'll lead the design, development, and deployment of enterprise-grade data architecture using Snowflake and a modern suite of data tools. You'll work at the intersection of IT and business, collaborating closely with stakeholders to create solutions that unlock the full value of data—driving decision-making and business growth across the organisation. Key Responsibilities Architecture Design: Design and maintain enterprise data architecture models, data flows, and integrations using Snowflake. Data Governance: Manage data collection, storage, transformation, and protection across systems. Database Development: Create More ❯

Title: Azure Data Engineer Location: Fully remote (working UAE hours) Salary: 5k GBP per month Are you a data expert with a passion for building scalable, cloud-first data pipelines? We’re hiring an experienced Azure Data Engineer to join a fast … growing team focused on developing a centralised, enterprise-grade data warehouse in the Azure cloud. This is a full-time remote role open to candidates located within GMT +1 to +6 time zones . If you thrive in fast-paced environments, take extreme ownership of your work, and … Creating and maintaining secure, scalable data pipelines and ingestion workflows Leveraging Azure Synapse Analytics and ADF to deliver robust ELT solutions Ensuring pipeline execution, availability, and reliability across platforms Owning physical data models and collaborating with BI teams on reporting layers Supporting data More ❯

Title: Azure Data Engineer Location: Fully remote (working UAE hours) Salary: 5k GBP per month Are you a data expert with a passion for building scalable, cloud-first data pipelines? We’re hiring an experienced Azure Data Engineer to join a fast … growing team focused on developing a centralised, enterprise-grade data warehouse in the Azure cloud. This is a full-time remote role open to candidates located within GMT +1 to +6 time zones . If you thrive in fast-paced environments, take extreme ownership of your work, and … Creating and maintaining secure, scalable data pipelines and ingestion workflows Leveraging Azure Synapse Analytics and ADF to deliver robust ELT solutions Ensuring pipeline execution, availability, and reliability across platforms Owning physical data models and collaborating with BI teams on reporting layers Supporting data More ❯

Title: Azure Data Engineer Location: Fully remote (working UAE hours) Salary: 5k GBP per month Are you a data expert with a passion for building scalable, cloud-first data pipelines? We’re hiring an experienced Azure Data Engineer to join a fast … growing team focused on developing a centralised, enterprise-grade data warehouse in the Azure cloud. This is a full-time remote role open to candidates located within GMT +1 to +6 time zones . If you thrive in fast-paced environments, take extreme ownership of your work, and … Creating and maintaining secure, scalable data pipelines and ingestion workflows Leveraging Azure Synapse Analytics and ADF to deliver robust ELT solutions Ensuring pipeline execution, availability, and reliability across platforms Owning physical data models and collaborating with BI teams on reporting layers Supporting data More ❯

vital role in owning the Databricks strategy and how they use it to innovate across their industry-leading brands. Designing and developing robust data frameworks, powering real-time data pipelines, and creating data services that deliver actionable insights to their global customer base. Role … with business goals Architectural Leadership: Serve as the owner and responsible person for all Databricks- related activities, acting as the architect across the Data Engineering, Data Analytics, and Data Science teams. Collaboration Enablement: Bridge the gap between teams by setting up templates and frameworks … for seamless collaboration, data sharing, and enablement aligned with the product vision. Unity Catalogue Management: Manage Unity catalogue, including permissions, usage, paths, locations, and implementation, ensuring data accessibility, security, and reliability. Guidance and Training: Provide guidance, support, and training to teams regarding Databricks usage, offering best More ❯

Senior Data Analyst Sports Agency £600 per day (Outside IR35) 4 Days per week in London Office Role and Responsibilities Stakeholder Management : Build strong relationships with internal and external stakeholders, ensuring clear communication and project alignment. Project Management : Work on multiple projects simultaneously, delivering results on time and … maintaining quality in fast-paced environments Data Integration : Oversee data integration into CRM systems, ensuring smooth data flow and accessibility for analysis. Data Pipelines : Develop and optimise data pipelines, ensuring efficient data flow between systems for reporting and … insights. Data Strategy : Shape and execute data strategy, providing actionable insights to drive business decisions. Collaboration & Reporting : Work cross-functionally, preparing and presenting reports to guide business strategy and project outcomes. Essential tools - Power BI Python BigQuery Google Cloud Platform Looker Google Analytics 4 (GA4) Solid More ❯

Senior Data Analyst Sports Agency £600 per day (Outside IR35) 4 Days per week in London Office Role and Responsibilities Stakeholder Management : Build strong relationships with internal and external stakeholders, ensuring clear communication and project alignment. Project Management : Work on multiple projects simultaneously, delivering results on time and … maintaining quality in fast-paced environments Data Integration : Oversee data integration into CRM systems, ensuring smooth data flow and accessibility for analysis. Data Pipelines : Develop and optimise data pipelines, ensuring efficient data flow between systems for reporting and … insights. Data Strategy : Shape and execute data strategy, providing actionable insights to drive business decisions. Collaboration & Reporting : Work cross-functionally, preparing and presenting reports to guide business strategy and project outcomes. Essential tools - Power BI Python BigQuery Google Cloud Platform Looker Google Analytics 4 (GA4) Solid More ❯

Job Title: Senior Data Engineer Location: London or Telford Contract: 6 months Clearance Requirement: Must hold or be eligible to obtain SC clearance About the Role We are seeking a Senior Data Engineer to join our team on a permanent basis, working on a high-profile … UK Government programme. You'll apply your expertise in data engineering and software development to solve complex data challenges, supporting critical infrastructure and decision-making through scalable, secure, and innovative data solutions. This is a fantastic opportunity to work with modern technologies in a … collaborative and mission-driven environment, with the freedom to innovate and grow your skills. Key Responsibilities Design and lead implementation of data pipelines from operational systems to BI and analytics platforms Build, maintain, and operate scalable data environments and real-time data streaming systems More ❯

Full stack Microsoft Azure Data Engineer/Developer 7 months Remote Inside IR35 - Umbrella only Role Overview: We are seeking a hands-on Microsoft Databricks Developer with expertise in data engineering best practices. The ideal candidate will have strong experience in Git/DevOps integration, Unity … Catalog, and performance optimization for Databricks data transformation workloads. Key Responsibilities: Develop and optimize Databricks data pipelines for efficient data transformation. Implement code version management using Git and DevOps integration. Configure and manage Unity Catalog for secure and scalable data governance. Enhance … performance optimization in Databricks workloads, ensuring efficient resource utilization. Collaborate with data engineers and architects to design scalable solutions. Ensure adherence to best practices in Databricks data engineering. Required Skills & Experience: Hands-on experience with Microsoft Databricks and Apache Spark. Strong knowledge of Git, DevOps integration More ❯

Role: Senior Software Engineer , Data Solutions Location: Hybrid (2 days on-site in Central London) Salary: £70,000 to £80,000 + Excellent Benefits Candidates must be eligible for UK Security Clearance The Opportunity We are proud to be working in partnership with a major defence systems integrator … into a critical delivery team. This is a unique opportunity to join a high-performing, mission-led environment focused on building robust, scalable data and analytics platforms for government and defence customers. This hybrid role offers the chance to work on-site in London just two days a … week initially, with increased flexibility over time. You'll be part of a team delivering impactful solutions in a secure and complex data landscape—an ideal environment for those who want to develop cutting-edge systems and contribute to national resilience. The Role As a Senior Software Engineer More ❯

into a critical delivery team. This is a unique opportunity to join a high-performing, mission-led environment focused on building robust, scalable data and analytics platforms for government and defence customers. This hybrid role offers the chance to work on-site in London just two days a … week initially, with increased flexibility over time. You'll be part of a team delivering impactful solutions in a secure and complex data landscape—an ideal environment for those who want to develop cutting-edge systems and contribute to national resilience. The Role As a Senior Frontend Developer … you will play a pivotal role in the design, development, and operation of innovative front-end solutions that interface with enterprise-scale data ecosystems. Working in close collaboration with data engineers, delivery managers, external partners, and client stakeholders, you will help drive the transformation of critical More ❯

Responsibilities: Define and enforce Python development best practices, ensuring scalable and maintainable code. Develop secure, efficient Python-based tools and frameworks, including APIs, data pipelines, and client-facing applications. Collaborate with business and technical teams to understand requirements and deliver effective technical solutions. Drive automation and reusable components More ❯

party vendors (enterprise software solutions such Document Management, Time Management, Practice Management, Core business i.e. Microsoft etc) to support their integration and master data management implementations Develop Azure-based integration solutions using Logic Apps, Service Bus, Function Apps, API Management, and Data Factory (ADF). Build … robust data pipelines and ETL solutions that support business processes and reporting needs. Conduct code reviews on other people's solutions Working within a team of architects, data engineers, data analysts and C#/.Net Core developers the Integration Developer/Azure Integration Developer … optimises their technology environment. As such you will bring a mix of skills including: Expertise in Azure Integration Services suite with strong Azure Data Factory ETL Logic Apps, Service Bus, API Management Very strong SQL skills, including database design, stored procedures, and query optimisation Underlying programming expertise in More ❯

create modern systems by using Microsoft Fabric Act as a consultant, understanding the client requirements and deliver the best solutions. Building and optimising data pipelines. Debugging issues Keep costs under control Skills Microsoft Fabric - Lakehouse and warehouse models Azure Tools (Data Factory, Synapse, SQL) Python experience … for programming Databases, SQL and NoSQL (CosmosDB, KQL) Data Modelling: Kimball frameworks and 3NF Nice To Have Databricks Power BI AI/ML Azure Infrastructure DevOps Certifications Microsfot certified: fabric Analytics Engineer Associate DP-203 Azure data Engineering AZ-305 Azure Solutions Architect. If this sounds More ❯

Leading Financial Services client is currently going through a large Data transformation and is looking for Data Solution Manager to join them on this exciting journey. The roles are offering a base between £80,0000 to £85,000pa + a strong benefits package and flexible working. … One day in office in every 3 months in East Sussex As a Data Solution Manager you will report to our Head of Data Delivery and lead a small team of some of the best engineers in the industry to deliver, improve and maintain robust data pipelines within our ambitious data architecture. Requirements: You will have a strong background in data engineering, with experience in managing and mentoring other engineers. You should have significant experience with Snowflake and Microsoft Azure, particularly Azure Data Factory and Azure Dev Ops. More ❯

Lead Software Engineer – Cloud & Data Orchestration 📍 Hybrid (1–2 days/week in office) | 💼 £140k+ Package | 🕵️‍♂️ Security-Cleared or Security Clearable required Location: London | Sector: Intelligence, Analytics & Secure Data | Process: 3 stages Our client is a mission-driven consultancy delivering high-impact intelligence, research, and technical … re hiring a Lead Software Engineer/Founding Engineer to take the reins of a specialist engineering unit delivering microservice-based cloud and data solutions at scale. This is a hands-on technical leadership role where you'll work closely with the CTO, shape architecture and tooling, and … the future of our engineering capability. 🔍 What You’ll Be Doing Leading a growing team (currently 2 engineers + external partners) across complex data and cloud engineering projects Designing and delivering distributed solutions on an AWS-centric stack, with open-source flexibility Working with Databricks, Apache Iceberg, and More ❯

Lead Software Engineer – Cloud & Data Orchestration 📍 Hybrid (1–2 days/week in office) | 💼 £140k+ Package | 🕵️‍♂️ Security-Cleared or Security Clearable required Location: London | Sector: Intelligence, Analytics & Secure Data | Process: 3 stages Our client is a mission-driven consultancy delivering high-impact intelligence, research, and technical … re hiring a Lead Software Engineer/Founding Engineer to take the reins of a specialist engineering unit delivering microservice-based cloud and data solutions at scale. This is a hands-on technical leadership role where you'll work closely with the CTO, shape architecture and tooling, and … the future of our engineering capability. 🔍 What You’ll Be Doing Leading a growing team (currently 2 engineers + external partners) across complex data and cloud engineering projects Designing and delivering distributed solutions on an AWS-centric stack, with open-source flexibility Working with Databricks, Apache Iceberg, and More ❯

Lead Software Engineer – Cloud & Data Orchestration 📍 Hybrid (1–2 days/week in office) | 💼 £140k+ Package | 🕵️‍♂️ Security-Cleared or Security Clearable required Location: London | Sector: Intelligence, Analytics & Secure Data | Process: 3 stages Our client is a mission-driven consultancy delivering high-impact intelligence, research, and technical … re hiring a Lead Software Engineer/Founding Engineer to take the reins of a specialist engineering unit delivering microservice-based cloud and data solutions at scale. This is a hands-on technical leadership role where you'll work closely with the CTO, shape architecture and tooling, and … the future of our engineering capability. 🔍 What You’ll Be Doing Leading a growing team (currently 2 engineers + external partners) across complex data and cloud engineering projects Designing and delivering distributed solutions on an AWS-centric stack, with open-source flexibility Working with Databricks, Apache Iceberg, and More ❯