1 to 25 of 301 Data Pipeline Jobs in London

that solves real-world problems for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on … re underpinned by over 300 engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data architectures such as Data … data engineering projects. You will work closely with cross-functional teams and contribute to the strategic direction of our data initiatives. RESPONSIBILITIES Data Pipeline Development: Lead the design, implement, and maintain scalable data pipelines for ingesting, processing, and transforming large volumes of data from various sources using tools such More ❯

Role: Technical Data Architect Location: UK (remote is fine, candidate must be in UK and available for any IMP meeting) Contract: 3 months (will likelyextend) Data Feed Factory - for Telco Network Function Expectations on the role : Hands on Experience on design & architecture of the Data Platform, Technical components and Data Engg. related … Data management. This includes assessing data quality, accessibility, data contract and lineage including any other potential capability evaluation Analyze existing Data pipeline and data workflows to determine the integration points and any design considerations/framework and patterns for the data factory as per the need for … the business users including any ad hoc demand for Data Processing. Support the Automation Code samples and examples produced during MVP phase to address Data pipeline build including Data Platform capabilities More ❯

uniquely collaborative model across the firm and throughout all levels of the client organization, generating results that allow our clients to thrive. What You'll Do Join the Data Layer Team, a global portfolio transforming our organization into a data-driven enterprise! The Data Layer Team is a portfolio of 30 people who build … essential data platforms, products, and capabilities to empower our clients and colleagues with high-quality, actionable insights. Our focus is on creating scalable data solutions and advancing our data infrastructure to drive informed decision-making across the company. As Use Case Enablement Product Analyst within BCG's Data Layer Team, you will … collaborate with Use Case Enablement Product Owner and cross-functional teams to gather and analyze business and data requirements. Your role is critical to bridging the gap between business stakeholders and technical teams, ensuring that new GenAI use cases are well-scoped, feasible, and aligned with user needs. You will work with various Gen AI use cases and More ❯

skills and experience, we're looking for people who are innovative, commercial and value the work that they do. Our London office is currently looking for a Senior Data Engineer. The Role The Senior Data Engineer will be responsible for designing, developing, and maintaining the infrastructure and systems required for data storage, processing, and … analysis. They play a crucial role in building and managing the data pipelines that enable efficient and reliable data integration, transformation, and delivery for all data users across the enterprise. Key Responsibilities Designs and develops data pipelines that extract data from various sources, transform it into the desired format, and … load it into the appropriate data storage systems Collaborates with data scientists and analysts to optimize models and algorithms for data quality, security, and governance Integrates data from different sources, including databases, data warehouses, APIs, and external systems Ensures data consistency and integrity during the integration process, performing More ❯

As a Data Engineer/Scientist at Actica, you will have the opportunity to design, build, and maintain data pipelines while developing advanced analytics solutions to unlock business problems for high-profile UK public sector organisations. Your expertise will enable organisations to maximise the value of their data assets through robust data … a key role in nationally critical projects that make a real difference to people's everyday lives. Locations: London, Guildford, Bristol, M4 corridor Hybrid working Actica recognises that data engineering, analytics, and data science are distinct but interconnected disciplines. Our data engineers focus on building and maintaining the data infrastructure that enables … analytics and data science work, while our data scientists and analysts focus on deriving insights and developing models. At Actica, we're at the forefront of the UK government's AI transformation agenda, seeking skilled data specialists to architect and implement advanced analytics solutions for critical public sector and defence organisations. As part of More ❯

This newly created Data Engineer Managers position is an excellent opportunity for a hands-on Senior Data Engineer/Technical Lead/Engineering Team Lead etc looking to move into a management position. The postholder will lead and mentor a team of 3 Data Engineers whilst overseeing data platform development and optimisation. … Our client is a well-established and rapidly growing global ecommerce business with its headquarters based in London. The Data Engineer Manager will play a pivotal role at the heart of our clients data & analytics operation. Having implemented a new MS Fabric based Data platform, the need is now to scale up and meet … the demand to deliver data driven insights and strategies right across the business globally. Therell be a hands-on element to the role as you'll be troubleshooting, doing code reviews, steering the team through deployments and acting as the escalation point for data engineering. This is a hybrid role based in Central/West London More ❯

DATA ARCHITECT - DV CLEARED NEW PERMANENT JOB OPPORTUNITY AVAILABLE WITHIN A LEADING NATIONAL SECURITY SME FOR A DATA ARCHITECT WITH ENHANCED DV CLEARANCE Permanent job opportunity for a Data Architect Leading National Security SME Salary up to £100,000 + Bonus London based organisation in an easily accessible location To apply please call or … email WHO WE ARE? We are recruiting multiple a Data Architect to support urgent National Security & Defence projects in London. Due to the nature of these projects you must hold enhanced DV Security Clearance. WHAT WILL THE DATA ARCHITECT BE DOING? You will be joining a leading SME who is working hard to support National Security … projects within UK Govt. Departments in London. As part of a team, you will be responsible for designing and implementing Data Solutions in Mission-Critical areas. WE NEED THE DATA ARCHITECT TO HAVE.... Current DV clearance - Enhanced Good at understanding complexity and abstracting that into a form that is consumable for a non-technical audience. Experience More ❯

global advisory, broking, and solutions company. We work with clients across a wide range of industries, helping them manage risk, optimise benefits, and improve performance. As a Fabric Data Engineer, you will play a key role in leveraging Microsoft Fabric, Azure, and Python to design and build advanced data solutions in the insurance domain. Location: London … UK Role: Hybrid Workstyle (Full-time) Role Overview: As a Fabric Data Engineer at WTW, you will take ownership of developing and optimising data pipelines, workflows, and ETL processes. You will work with cutting-edge technologies to ensure that data is efficiently processed, stored, and made accessible for analysis. This role is a key … processes using Microsoft Fabric or Azure technologies. Manage and optimise notebooks, pipelines, and workflows to enhance the performance and efficiency of our data architecture. Data Pipeline Development & ETL : Build and maintain high-quality ETL pipelines to clean, transform, and enrich data from various sources. Ensure that pipelines are automated, scalable, and fault-tolerant More ❯

Join our rapidly expanding team as a hands-on Cloud Data Analytics Platform Engineer and play a pivotal role in shaping the future of data at Citi. We're building a cutting-edge, multi-cloud data analytics platform that empowers our users with secure, scalable, and efficient data insights. This role sits … at the intersection of infrastructure, data engineering, and architecture, offering a unique opportunity to work with the latest cloud-native technologies and influence our data strategy. This is a hands-on role requiring deep technical skills and a passion for building and optimizing data platforms. What You'll Do: Architect and Build: Design and … data zones. Familiarity with data governance tools and frameworks. IaC Proficiency: Solid experience with Terraform and preferably Harness, Tekton, or Lightspeed for CI/CD pipeline management. Kubernetes Mastery: Strong command of Kubernetes, especially in the context of data processing workloads. Security Focus: A firm grasp of cloud security principles and best practices. More ❯

private and commercial banking, wealth planning and investment management. We believe in traditional relationship and service-led banking powered by modern technology. Job purpose Develop and maintain complex data systems that support the Bank's operations and reporting needs, ensuring they meet all essential data requirements, following strong engineering and automation best practices and collaborating with … Significant hands-on experience with the Azure Data Stack, critically ADF and Synapse (experience with Microsoft Fabric is a plus) Highly developed python and data pipeline development knowledge, must include substantial PySpark experience Demonstrable DevOps and DataOps experience with an understanding of best practices for engineering, test and ongoing service delivery An understanding of Infrastructure … as Code concepts (Demonstrable Terraform experience a plus) Demonstrable experience of Data Pipeline testing, including automated testing, data validation and code assurance Demonstrable experience of working within Agile Delivery projects An understanding of data formats for ingest, transformation and analytics, data security, access control and authorisation, GDPR, data privacy More ❯

Senior Data Engineer Location: London, UK (3 days in the office) SC Cleared: Required Essential Skills & Experience: 10+ years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks. Strong proficiency in Python and Spark (PySpark) or Scala. Deep understanding of data warehousing principles, data modelling techniques, and data integration patterns. Extensive experience with Azure data services, including Azure Data Factory … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

Take a look at our current vacancies. If you see a vacancy that is right for you, we encourage you to apply! Job Title: Specialist Data Engineer Full-time role Working Hours : Full-time 35 hours per week Salary: £64,575 Location: London Docklands (Hybrid). Our permanent hybrid policy sees us all working at least four days … we have already delivered significant improvements in the service we offer and 2026 will see us continue with our ambitious transformation agenda.This year, we created our new Chief Data Office as part of redefining our data strategy across the organisation. With data playing an increasingly vital role in our transformation journey, the CDO is … shaping a modern, cloud-first data landscape that supports innovation, enhances decision-making, and strengthens our customer-centric approach. It's an exciting time to be part of the Data team at the Financial Ombudsman Service Key Responsibilities: Designing and maintaining ETL/ELT pipelines using Azure Data Factory, Synapse and Databricks. Collaborating with More ❯

AWS Data Engineer - VP Level Location: London (Canary Wharf)Salary: £115000 - 120000 A leading financial institution is seeking a Senior AWS Data Engineer to join a greenfield initiative focused on transforming market data access and utilization. This VP-level role offers strategic influence, technical leadership, and cross-functional collaboration across business units. Key Responsibilities … Design and maintain scalable data pipelines, warehouses, and lakes using AWS technologies.* Implement robust data governance, quality, and security practices.* Collaborate with data scientists to deploy machine learning models.* Lead or contribute to strategic planning, risk management, and stakeholder engagement.* Drive innovation through advanced analytics and research-based problem solving. To be successful you … should have: 10 years hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Previous experience in implementing best practices for data engineering, including data governance, data quality, and data security. Proficiency in data More ❯

AWS Data Engineer - VP Level Location: London (Canary Wharf) Salary: £(phone number removed) - (phone number removed) A leading financial institution is seeking a Senior AWS Data Engineer to join a greenfield initiative focused on transforming market data access and utilization. This VP-level role offers strategic influence, technical leadership, and cross-functional collaboration across … business units. Key Responsibilities: * Design and maintain scalable data pipelines, warehouses, and lakes using AWS technologies. * Implement robust data governance, quality, and security practices. * Collaborate with data scientists to deploy machine learning models. * Lead or contribute to strategic planning, risk management, and stakeholder engagement. * Drive innovation through advanced analytics and research-based problem solving. … To be successful you should have: 10 years hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Previous experience in implementing best practices for data engineering, including data governance, data quality, and data security. Proficiency More ❯

Data Engineer Location: Hammersmith - Hybrid after probation Salary: £38,000 -£45,000 Join Our Team as a Data Engineer! About Us : Join a forward-thinking organisation dedicated to revolutionising retail through innovative solutions and operations. We leverage physical spaces to test new concepts and pilot initiatives, helping brands and retailers thrive in a dynamic marketplace. What … You'll Do : As our Data Engineer, you will play a pivotal role in the day-to-day management and maintenance of our data infrastructure, driving our large-scale programmes in the USA. Your primary responsibilities will include : Design & Development: Create, develop, and maintain scalable data pipelines and ETL processes across various systems. Data … star schema, data vault, dimensional modelling). Proficiency in Excel-based data workflows for various Agile Retail projects. Hands-on experience with data pipeline orchestration tools (Airflow, dbt, Prefect, or similar). Benefits: Unlimited holiday Annual Wellbeing Allowance Flexible work culture Monthly socials and events Complimentary snack bar Employer pension contribution If you More ❯

Data Engineer Location: Hammersmith - Hybrid after probation Salary: £38,000 -£45,000 Join Our Team as a Data Engineer! About Us : Join a forward-thinking organisation dedicated to revolutionising retail through innovative solutions and operations. We leverage physical spaces to test new concepts and pilot initiatives, helping brands and retailers thrive in a dynamic marketplace. What … You'll Do : As our Data Engineer, you will play a pivotal role in the day-to-day management and maintenance of our data infrastructure, driving our large-scale programmes in the USA. Your primary responsibilities will include : Design & Development: Create, develop, and maintain scalable data pipelines and ETL processes across various systems. Data … star schema, data vault, dimensional modelling). Proficiency in Excel-based data workflows for various Agile Retail projects. Hands-on experience with data pipeline orchestration tools (Airflow, dbt, Prefect, or similar). Benefits: Unlimited holiday Annual Wellbeing Allowance Flexible work culture Monthly socials and events Complimentary snack bar Employer pension contribution If you More ❯

Job Title: Data Specialist Role Summary: We are looking for a detail-oriented Data Specialist to manage, analyze, and maintain data across enterprise systems. The role involves ensuring data accuracy, integrity, and accessibility while supporting business intelligence, reporting, and decision-making processes. The Data Specialist will collaborate with cross-functional … teams to align data management practices with business needs. Key Responsibilities: Collect, clean, validate, and maintain large volumes of structured and unstructured data. Ensure data accuracy, consistency, and integrity across databases and systems. Support data extraction, transformation, and loading (ETL) processes. Develop and maintain reports, dashboards, and data visualizations to support business … insights. Collaborate with business analysts, data engineers, and stakeholders to define data requirements. Monitor data pipelines and resolve data quality or processing issues. Implement and enforce data governance, security, and compliance policies. Contribute to process improvements in data management and reporting workflows. Required Skills & Experience: 3 6+ years More ❯

Job Title: Data Specialist Role Summary: We are looking for a detail-oriented Data Specialist to manage, analyze, and maintain data across enterprise systems. The role involves ensuring data accuracy, integrity, and accessibility while supporting business intelligence, reporting, and decision-making processes. The Data Specialist will collaborate with cross-functional … teams to align data management practices with business needs. Key Responsibilities: Collect, clean, validate, and maintain large volumes of structured and unstructured data. Ensure data accuracy, consistency, and integrity across databases and systems. Support data extraction, transformation, and loading (ETL) processes. Develop and maintain reports, dashboards, and data visualizations to support business … insights. Collaborate with business analysts, data engineers, and stakeholders to define data requirements. Monitor data pipelines and resolve data quality or processing issues. Implement and enforce data governance, security, and compliance policies. Contribute to process improvements in data management and reporting workflows. Required Skills & Experience: 3–6+ years More ❯

Data Analytics Engineer Location: Central London - hybrid working Rate: £500.00 per day inside IR35 Duration: 12 months We are working with a leading global player in the travel industry who are seeking a Data Analytics Engineer to join their data team. This role sits at the intersection of data analysis and data engineeringideal for someone who enjoys building scalable data solutions while staying close to business impact. The Role As a Data Analytics Engineer , youll design, build, and maintain reliable data pipelinesprimarily using PySpark, SQL, and Python to ensure business teams (analysts, product managers, finance, operations) have access to well-modeled, actionable data. Youll … Engineers, youll be closer to the business, shaping analytics-ready datasets. Unlike pure Data Analysts, youll spend more time coding, managing data infrastructure, and ensuring pipeline reliability. Who Were Looking For Data Analytics : Analysts who have strong experience building and maintaining data pipelines (particularly in PySpark/SQL ) and want to More ❯

We are looking to hire a additional 2 Data Engineers to join the Satalia Data Engineering Team. We are investing massively in developing next-generation AI tools for multimodal datasets and a wide range of applications. We are building large scale, enterprise grade solutions and serving these innovations to our clients and WPP agency partners. As … Your contributions will directly shape cutting-edge AI products and services that make a tangible impact for FTSE 100 clients. What you’ll be doing: Collaborate closely with data scientists, architects, and other stakeholders to understand and implement business requirements. Provide data engineering support for AI model development and deployment, ensuring data scientists have … access to the data they need in the format they need it. Implement and optimize data transformations and ETL/ELT processes, using appropriate data engineering tools. Work with a variety of databases and data warehousing solutions to store and retrieve data efficiently. Implement monitoring, troubleshooting, and maintenance procedures for More ❯

and science capabilities for ROW Ops. We develop scalable analytics applications, AI/ML products and research models to optimize operation processes. You will work with Product Managers, Data Engineers, Data Scientists, Research Scientists, Applied Scientists and Business Intelligence Engineers using rigorous quantitative approaches to ensure high quality data/science products for our … customers around the world. As a Data Engineer, you will play a crucial role in supporting the team by creating and maintaining the data infrastructure necessary for the advanced analytics and machine learning solutions. Our team solves a broad range of problems that can be scaled across ROW (Rest of the World including countries like India … Route for $MM savings • Deep Learning models to synthesize attributes of addresses • Abuse detection models to reduce network losses Key job responsibilities 1. Design, develop, and maintain scalable data pipelines to support ML model development and production deployment. 2. Implement and maintain CI/CD pipelines for the data and ML solutions. 3. Collaborate with data More ❯

Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Job Description: We are looking for a seasoned AWS Data Engineer to join our team and contribute to the development of scalable, secure, and high-performance data solutions. This role involves modernizing mainframe legacy systems, building robust … data pipelines, and enabling advanced analytics using AWS-native technologies. You’ll collaborate with cross-functional teams and play a key role in shaping our data strategy. Skills Must have : Proven experience in designing and maintaining scalable data lakes using AWS S3, with strong understanding of data lifecycle, access control, and performance … optimization. Hands-on experience migrating data from Mainframes to AWS, including transformation and validation of legacy formats. Proficiency in reading and processing VSAM data using Copybooks and the Cobrix library. Experience implementing data retention and archival strategies in cloud environments. Strong understanding and practical implementation of Medallion Architecture (Bronze, Silver, Gold layers) for structured More ❯

Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Job Description: We are looking for a seasoned AWS Data Engineer to join our team and contribute to the development of scalable, secure, and high-performance data solutions. This role involves modernizing mainframe legacy systems, building robust … data pipelines, and enabling advanced analytics using AWS-native technologies. You’ll collaborate with cross-functional teams and play a key role in shaping our data strategy. Skills Must have : Proven experience in designing and maintaining scalable data lakes using AWS S3, with strong understanding of data lifecycle, access control, and performance … optimization. Hands-on experience migrating data from Mainframes to AWS, including transformation and validation of legacy formats. Proficiency in reading and processing VSAM data using Copybooks and the Cobrix library. Experience implementing data retention and archival strategies in cloud environments. Strong understanding and practical implementation of Medallion Architecture (Bronze, Silver, Gold layers) for structured More ❯

Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Job Description: We are looking for a seasoned AWS Data Engineer to join our team and contribute to the development of scalable, secure, and high-performance data solutions. This role involves modernizing mainframe legacy systems, building robust … data pipelines, and enabling advanced analytics using AWS-native technologies. You’ll collaborate with cross-functional teams and play a key role in shaping our data strategy. Skills Must have : Proven experience in designing and maintaining scalable data lakes using AWS S3, with strong understanding of data lifecycle, access control, and performance … optimization. Hands-on experience migrating data from Mainframes to AWS, including transformation and validation of legacy formats. Proficiency in reading and processing VSAM data using Copybooks and the Cobrix library. Experience implementing data retention and archival strategies in cloud environments. Strong understanding and practical implementation of Medallion Architecture (Bronze, Silver, Gold layers) for structured More ❯

Media, Retail and CPG, and Public Services. Consolidated revenues as of 12 months ending December 2024 totaled $13.8 billion. Job Description: We are looking for a seasoned AWS Data Engineer to join our team and contribute to the development of scalable, secure, and high-performance data solutions. This role involves modernizing mainframe legacy systems, building robust … data pipelines, and enabling advanced analytics using AWS-native technologies. You’ll collaborate with cross-functional teams and play a key role in shaping our data strategy. Skills Must have : Proven experience in designing and maintaining scalable data lakes using AWS S3, with strong understanding of data lifecycle, access control, and performance … optimization. Hands-on experience migrating data from Mainframes to AWS, including transformation and validation of legacy formats. Proficiency in reading and processing VSAM data using Copybooks and the Cobrix library. Experience implementing data retention and archival strategies in cloud environments. Strong understanding and practical implementation of Medallion Architecture (Bronze, Silver, Gold layers) for structured More ❯