1 to 25 of 763 Data Pipeline Jobs in London

engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources … effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote … SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality More ❯

engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation: Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources … effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote … SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality More ❯

advantage, and its technology-led service provides access to all major exchanges, order-flow management via screen, voice and DMA, plus award-winning data, insights and analytics. The Technology Department delivers differentiation, scalability and security for the business. Reporting to the COO, Technology provides digital tools, software services … our enterprise-wide services to end users and actively manages the firm's infrastructure and data. Within IT, Marex Technology has established a Data team that enables the firm to leverage data assets to increase productivity and improve business decisions, as well as maintain data compliance. The Data Team encompasses Database Administration, Data Engineering, Data Analysis, Data Architecture, Data Intelligence and AI expertise. In recent years, they have developed a Data Lakehouse architecture, that is relied upon by different departments across More ❯

advantage, and its technology-led service provides access to all major exchanges, order-flow management via screen, voice and DMA, plus award-winning data, insights and analytics. The Technology Department delivers differentiation, scalability and security for the business. Reporting to the COO, Technology provides digital tools, software services … our enterprise-wide services to end users and actively manages the firm's infrastructure and data. Within IT, Marex Technology has established a Data team that enables the firm to leverage data assets to increase productivity and improve business decisions, as well as maintain data compliance. The Data Team encompasses Database Administration, Data Engineering, Data Analysis, Data Architecture, Data Intelligence and AI expertise. In recent years, they have developed a Data Lakehouse architecture, that is relied upon by different departments across More ❯

get in touch with at UKI.recruitment@tcs.com or call TCS London Office number 02031552100 with the subject line: “Application Support Request”. Role : Data Architect Job Type: Permanent (Hybrid) Location: London, United Kingdom Ready to utilize your skills in designing, creating, and managing data architecture? Join … us as a Data Architect. Careers at TCS: It means more TCS is a purpose-led transformation company, built on belief. We do not just help businesses to transform through technology. We support them in making a meaningful difference to the people and communities they serve - our clients … an exciting team where you will be challenged every day. • Build strong relationships with a diverse range of stakeholders. The Role As a Data Architect, you will be responsible for designing, creating, and managing data architecture. You will also ensure that data is efficiently More ❯

a newly created role within the NAO's Digital Services (DS) function with responsibility for supporting the development and continual improvement of NAO data & technology service composition and provision. They will support emerging tech to enable the automation or acceleration of relevant NAO processes and derive deeper insights … from corporate and client data. In this capacity, you will transform organizational data into structured formats suitable for analysis and decision-making. You will develop and test data models, explore local data sources, and construct pipelines from corporate repositories to data science … and machine learning models. Acting as a domain-specific collaborator to data engineers, you will facilitate the conversion of data into actionable intelligence, thereby contributing to the NAO's commitment to data-driven excellence. In this role, you will: Collaborate with subject matter experts More ❯

Position : Senior Data Engineer Salary: £55,000 - £65,000 Location : Hybrid - Bristol, Manchester, or London Security Requirement : SC Clearance to start and be willing/able to obtain DV Clearance We are seeking Data Engineers, with a keen interest/experience in AI/ML and … in Python Scripting. To join our Consultancy client working on meaningful projects in the Defence and Security sector. Required Experience: End-to-End Data Development : Strong experience with data pipelines, ETL processes, and workflow orchestration, demonstrating best practices across tech stacks. Strong Python Skills Diverse Data … engineering tools, integration frameworks, and query engines to create high-quality, standardised data for AI applications and reporting. Data Pipeline Development : Design and implement robust data pipelines and stores in collaboration with other engineers and developers. Innovative Problem Solving : Bring fresh approaches More ❯

Select how often (in days) to receive an alert: Head of Data Strategy Location: UK - Hatfield, UK - London, UK - Manchester, UK - Milton Keynes, UK - Nottingham Job-ID: 213406 Contract type: Standard Business Unit: Information Technology Life on the team You will oversee the development and implementation of a … comprehensive data strategy, encompassing data governance, data management, data quality, and data analytics to ensure high-quality data outcomes, insights leading to action, maximizing data value, and treating data as an asset. What you … ll do As Head of Data Strategy, you will be responsible for defining and implementing our Data Strategy, including data governance, management, quality, master data management, and reporting/analytics/outcomes-aligned with our Technology Strategy. You will: Own, define, and More ❯

Data Engineer/Scientist Department: Consultancy Employment Type: Full Time Location: United Kingdom/Hybrid Description As a Data Engineer/Scientist at Actica, you will have the opportunity to design, build, and maintain data pipelines while developing advanced analytics solutions to unlock business … problems for high-profile UK public sector organisations. Your expertise will enable organisations to maximise the value of their data assets through robust data infrastructure and sophisticated analysis, playing a key role in nationally critical projects that make a real difference to people's everyday lives. … Locations: London, Guildford, Bristol, M4 corridor Hybrid working Roles and Responsibilities Actica recognises that data engineering, analytics, and data science are distinct but interconnected disciplines. Our data engineers focus on building and maintaining the data infrastructure that enables analytics and data More ❯

Job Title: Lead Data Engineer (AWS) Business Unit/Segment: Data Management/Analytics Location: London, United Kingdom (Flexible hybrid working) Employment Type: Permanent - Salary -£80k to £100k Summary of the Role: A leading global data and AI company is looking for a Lead … with emerging data technologies and recommend improvements to data engineering practices. Develop and enforce best practices for data pipeline orchestration, testing, and deployment. Provide mentorship, feedback, and leadership across project and operational initiatives. Collaborate with cross-functional teams to design a consistent and … CD practices and code versioning systems. Key Skills and Attributes: Cloud & Data Engineering: Deep expertise in AWS services and data pipeline development. Data Modelling & Architecture: Strong background in data warehousing and modern modelling frameworks. Leadership: Ability to lead teams, provide feedback More ❯

Job Title: Lead Data Engineer (AWS) Business Unit/Segment: Data Management/Analytics Location: London, United Kingdom (Flexible hybrid working) Employment Type: Permanent - Salary -£80k to £100k Summary of the Role: A leading global data and AI company is looking for a Lead … with emerging data technologies and recommend improvements to data engineering practices. Develop and enforce best practices for data pipeline orchestration, testing, and deployment. Provide mentorship, feedback, and leadership across project and operational initiatives. Collaborate with cross-functional teams to design a consistent and … CD practices and code versioning systems. Key Skills and Attributes: Cloud & Data Engineering: Deep expertise in AWS services and data pipeline development. Data Modelling & Architecture: Strong background in data warehousing and modern modelling frameworks. Leadership: Ability to lead teams, provide feedback More ❯

Job Title: Senior Data Engineer Location: Abu Dhabi Job Summary: As a Senior Data Engineer , you will be responsible for designing, developing, and maintaining advanced, scalable data systems that power critical business decisions. You will lead the development of robust data pipelines … role requires a deep understanding of modern data engineering practices, real-time processing, and cloud-native solutions. Key Responsibilities: Data Pipeline Development & Management: Design, implement, and maintain scalable and reliable data pipelines to ingest, transform, and load structured, unstructured, and real-time data … environments. Machine Learning & Advanced Analytics Enablement: Collaborate with data scientists to prepare and serve features for ML models. Maintain awareness of ML pipeline integration and ensure data readiness for experimentation and deployment. Documentation & Continuous Improvement: Maintain thorough documentation including technical specifications , data flow More ❯

Job Title: Senior Data Engineer Location: Abu Dhabi Job Summary: As a Senior Data Engineer , you will be responsible for designing, developing, and maintaining advanced, scalable data systems that power critical business decisions. You will lead the development of robust data pipelines … role requires a deep understanding of modern data engineering practices, real-time processing, and cloud-native solutions. Key Responsibilities: Data Pipeline Development & Management: Design, implement, and maintain scalable and reliable data pipelines to ingest, transform, and load structured, unstructured, and real-time data … environments. Machine Learning & Advanced Analytics Enablement: Collaborate with data scientists to prepare and serve features for ML models. Maintain awareness of ML pipeline integration and ensure data readiness for experimentation and deployment. Documentation & Continuous Improvement: Maintain thorough documentation including technical specifications , data flow More ❯

Data Engineer Inside IR35 Contract Up to £750 per day DV Clearance ESSENTIAL London We are seeking a highly skilled Data Engineer to join a prestigious defence organisation in London. This is an exciting opportunity to lead and mentor a junior Data Engineer while … pipelines and products. Key Responsibilities: Technical Leadership: Lead and guide the technical direction of data engineering projects. Data Pipeline Development: Design, develop, and maintain robust data pipelines using Azure Data Factory (ADF), Azure Synapse Analytics, and SQL Server. Data … teams to ensure seamless integration and effective communication. Essential Skills and Experience: Proven experience in data engineering roles, including data pipeline development and data integration. Strong proficiency in Azure Data Factory (ADF), Azure Synapse Analytics, and SQL Server. Solid understanding of More ❯

Data Engineer Inside IR35 Contract Up to £750 per day DV Clearance ESSENTIAL London We are seeking a highly skilled Data Engineer to join a prestigious defence organisation in London. This is an exciting opportunity to lead and mentor a junior Data Engineer while … pipelines and products. Key Responsibilities: Technical Leadership: Lead and guide the technical direction of data engineering projects. Data Pipeline Development: Design, develop, and maintain robust data pipelines using Azure Data Factory (ADF), Azure Synapse Analytics, and SQL Server. Data … teams to ensure seamless integration and effective communication. Essential Skills and Experience: Proven experience in data engineering roles, including data pipeline development and data integration. Strong proficiency in Azure Data Factory (ADF), Azure Synapse Analytics, and SQL Server. Solid understanding of More ❯

Technical Data Specialist Contract onsite in London Day rate: £650 - £858 per day inside IR35 We're currently seeking an accomplished and highly ambitious Technical Data Specialist to work with our exceptional client, a world class brand in a secure environment to drive major business transformation … process and technical change across this complex organisation. This contract roles will require you to conduct security clearance prior to assignment. The Technical Data Specialist role is conducted in an environment that is far from ordinary, therefore, we're not looking for ordinary. The Technical Data Specialists set the vision for the organisation's use of data (through data design) to ensure that data is managed properly and meets the organisation's needs. You will work with key internal and external partners to deliver data architecture More ❯

the complexity of modern cruising - all while keeping things collaborative and fun. Your Mission: Are you passionate about unlocking the power of customer data? We're looking for a talented Lead Data Engineer to spearhead the design, development, and optimisation of our critical CRM and customer … data transformation. You'll play a pivotal role in building the data foundations for advanced analytics, personalised customer experiences, and effective marketing activation. As a senior member of the team, you will lead data engineers and champion best practices within our data … data engineers, fostering their growth and ensuring high standards in coding, testing, and documentation. Optimise: Continuously monitor and improve data pipeline performance, reliability, and cost-efficiency. Document: Create and maintain clear, comprehensive documentation for data models, pipeline architecture, and processes. What You More ❯

who think big, act fearlessly, and create an inclusive environment that drives positive impact to join our team. What you'll do The Data Led Sales team is a central function within the Product Management division. We build software tools to help the sales teams in the business … organizations systems and processes. The team has a variety of skills and experience in technical domains such as Software development, Automation, Data-pipeline-modelling, IT project management, Data visualisation. We also pride ourselves on our close integration with non-technical/non-data … full back-end data architecture on behalf of the Data Led Sales team • Create and maintain optimal data pipeline architectures • Design and implement data quality monitoring and testing frameworks • Work with customers to define and apply data modeling guidelines More ❯

platforms that handle the complexity of modern cruising - all while keeping things collaborative and fun. Your Mission: Are you eager to build the data solutions that power exceptional customer experiences? We're looking for a motivated Data Engineer to join our team and focus on developing … and maintaining our critical CRM and customer data pipelines. Working closely with the Lead Data Engineer, you'll contribute directly to the data foundations that enable advanced analytics, personalisation, and effective marketing. This is a fantastic opportunity to grow your skills in a dynamic … pipelines and processes. What You'll Be Responsible For: The successful development, testing, deployment, and maintenance of assigned customer data pipeline components. Adhering to the technical quality, scalability, and reliability standards set for the customer data platform. Following data engineering best More ❯

Data Engineer | London Market Broker | London/Hybrid | £50 - 55,000 (doe) Are you passionate about designing and optimizing data infrastructures? Do you have a flair for working with large datasets and cloud technologies? We are seeking a highly skilled Data Engineer to join … our client, who are a boutique London market broker. As a Data Engineer, you will play a vital role in developing scalable data pipelines, ensuring data quality, and enabling efficient data processing to support business intelligence and advanced analytics. You will also … be involved in their Cloud migration project. Some key responsibilities include: Design, develop, and maintain scalable and reliable data pipelines. Build and optimize data architectures, data lakes, and data warehouses. Develop ETL/ELT processes to extract, transform, and load data More ❯

Data Engineer | London Market Broker | London/Hybrid | £50 - 55,000 (doe) Are you passionate about designing and optimizing data infrastructures? Do you have a flair for working with large datasets and cloud technologies? We are seeking a highly skilled Data Engineer to join … our client, who are a boutique London market broker. As a Data Engineer, you will play a vital role in developing scalable data pipelines, ensuring data quality, and enabling efficient data processing to support business intelligence and advanced analytics. You will also … be involved in their Cloud migration project. Some key responsibilities include: Design, develop, and maintain scalable and reliable data pipelines. Build and optimize data architectures, data lakes, and data warehouses. Develop ETL/ELT processes to extract, transform, and load data More ❯

Data Engineer (DV Cleared) 6 Month Contract (Inside IR35) Location: London Rate: £700 - £800 per day Skills: DV Clearance ESSENTIAL - Must already hold DV clearance, Cloud technology experience We are looking to recruit a DV Cleared Data Engineer to join a defence organisation based in Central … office based in Central London. Key Responsibilities: Technical Leadership: Lead and guide the technical direction of data engineering projects. Data Pipeline Development: Design, develop, and maintain robust data pipelines using Azure Data Factory (ADF), Azure Synapse Analytics, and SQL Server. Data … teams to ensure seamless integration and effective communication. Essential Skills and Experience: Proven experience in data engineering roles, including data pipeline development and data integration. Experience working with diverse data sets and data sources. Excellent communication and collaboration skills. More ❯

Data Engineer (DV Cleared) 6 Month Contract (Inside IR35) Location: London Rate: £700 - £800 per day Skills: DV Clearance ESSENTIAL - Must already hold DV clearance, Cloud technology experience We are looking to recruit a DV Cleared Data Engineer to join a defence organisation based in Central … office based in Central London. Key Responsibilities: Technical Leadership: Lead and guide the technical direction of data engineering projects. Data Pipeline Development: Design, develop, and maintain robust data pipelines using Azure Data Factory (ADF), Azure Synapse Analytics, and SQL Server. Data … teams to ensure seamless integration and effective communication. Essential Skills and Experience: Proven experience in data engineering roles, including data pipeline development and data integration. Experience working with diverse data sets and data sources. Excellent communication and collaboration skills. More ❯

and we're still growing. JOB PURPOSE The Senior Analytics Engineer leads the development and optimisation of our advanced analytics ecosystem, ensuring that data is not only fit for purpose but also robustly designed for scalability, reusability, and actionable insights. This role bridges technical expertise and strategic vision … making. Key responsibilities include designing scalable data architectures in conjunction with Domain and Product Data Architects, overseeing data pipeline development, and maintaining data integrity. This role involves collaboration with Domain and Business Data Stewards to align data … is demonstrated through rigorous validation and monitoring processes throughout the data pipeline. Strong problem-solving abilities are essential for diagnosing and resolving pipeline issues while optimising performance for scalability. Knowledge of analytics platforms, such as Tableau and ThoughtSpot, ensures compatibility with data visualisation and reporting More ❯

Data Governance Manager I am working with a hugely successful organisation based in London who are looking for a Data Governance Manager to join their newly formed data practice. You will be responsible for ensure the data used across the business in … reliable and governed correctly, whilst working closely with stakeholders across the wider organisation to implement data quality, governance and MDM frameworks. You will join a growing team where you will have the opportunity to take ownership of projects and shape the way the business uses its data. You … Management processes across the wider organisation Work closely with technical teams to embed data quality checks into data pipeline builds to ensure data is readily available for reporting Manage data governance tools and platforms To be successful in this More ❯