1 to 25 of 482 Data Pipeline Jobs in London

a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data … technical solutions. Communicate technical concepts effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote and implement DevOps best practices. … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data … technical solutions. Communicate technical concepts effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote and implement DevOps best practices. … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation: Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data … technical solutions. Communicate technical concepts effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote and implement DevOps best practices. … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation: Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data … technical solutions. Communicate technical concepts effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote and implement DevOps best practices. … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

We have a current opportunity for a Data Engineer on a permanent basis. The position will be based in London. For further information about this position please apply. The team: You'll be working in a wider Data Management team alongside data engineers, data analysts & data governance specialists and facing … to stakeholders in the front office including product teams, analysts and traders. Your responsibilities: - Design and develop data catalogue and metadata management processes from data pipeline - Engineering data solutions - Implement data quality standards - Establish data governance and ensure data is fit for purpose - Understanding of data security, data architecture - Working alongside end users, stakeholders and data product teams Technology skillset: - Experience in data engineering, data modelling and data analytics [programming languages: Python, SQL, Alteryx] - Data Visualisation experience [tools: PowerBI, Tableau or similar] - MetaData management and data cataloguing [tools: Collibra, Alation More ❯

uniquely collaborative model across the firm and throughout all levels of the client organization, generating results that allow our clients to thrive. What You'll Do Join the Data Layer Team, a global portfolio transforming our organization into a data-driven enterprise! The Data Layer Team is a portfolio of 30 people who build … essential data platforms, products, and capabilities to empower our clients and colleagues with high-quality, actionable insights. Our focus is on creating scalable data solutions and advancing our data infrastructure to drive informed decision-making across the company. As Use Case Enablement Product Analyst within BCG's Data Layer Team, you will … collaborate with Use Case Enablement Product Owner and cross-functional teams to gather and analyze business and data requirements. Your role is critical to bridging the gap between business stakeholders and technical teams, ensuring that new GenAI use cases are well-scoped, feasible, and aligned with user needs. You will work with various Gen AI use cases and More ❯

and health security Detect: use cutting edge environmental and biological surveillance to proactively detect and monitor infectious diseases and threats to health Analyse: use world-class science and data analytics to assess and continually monitor threats to health, identifying how best to control and mitigate the risks Respond: take rapid, collaborative and effective actions nationally and locally to … lead strong and sustainable global, national, regional and local partnerships designed to save lives, protect the nation from public health threats and reduce inequalities. Job overview The UKHSA Data Design & Architecture (DD&A) team is responsible for supporting data architecture and data solutions across the organisation. We are a small team of solution architects … data architects, and data modellers. We also work closely with other parts of the organisation who also are involved in development and maintenance of infrastructure and software, including Programme Teams, Engineering & Enablement, Surveillance Systems, Data Services & Quality, Genomics, Enterprise Architecture and the Development and Operations team. The DD&A team supervises and actively develops More ❯

IR35) Databricks Engineer needed with active SC Security Clearance for 6 Month Contract based in Central London (Hybrid). Developing a cutting-edge Azure Databricks platform for economic data modelling, analysis, and forecasting. Start asap ideally in October 2025. Hybrid Working - 2 days/week remote (WFH), and 3 days/week working on-site from the Central … London office. A chance to work with a leading global IT and Digital transformation business specialising in Government projects: In-depth Data Engineering + strong hands-on Azure Databricks expertise. Azure Data Services, Azure Data Factory, Azure Blob Storage + Azure SQL Database. Designing, developing, building + optimising data pipelines, implementing data … including data modelling techniques + data integration patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data More ❯

IR35) Databricks Engineer needed with active SC Security Clearance for 6 Month Contract based in Central London (Hybrid). Developing a cutting-edge Azure Databricks platform for economic data modelling, analysis, and forecasting. Start ASAP in Nov/Dec 2025. Hybrid Working - 2 days/week remote (WFH), and 3 days/week working on-site from the … Central London office. A chance to work with a leading global IT and Digital transformation business specialising in Government projects: In-depth Data Engineering + strong hands-on Azure Databricks expertise. Azure Data Services, Azure Data Factory, Azure Blob Storage + Azure SQL Database. Designing, developing, building + optimising data pipelines, implementing … including data modelling techniques + data integration patterns. Experience of working with complex data pipelines, large data sets, data pipeline optimization + data architecture design. Implementing complex data transformations using Spark, PySpark or Scala + working with SQL/MySQL databases. Experience with data More ❯

skills and experience, we're looking for people who are innovative, commercial and value the work that they do. Our London office is currently looking for a Senior Data Engineer. The Role The Senior Data Engineer will be responsible for designing, developing, and maintaining the infrastructure and systems required for data storage, processing, and … analysis. They play a crucial role in building and managing the data pipelines that enable efficient and reliable data integration, transformation, and delivery for all data users across the enterprise. Key Responsibilities Designs and develops data pipelines that extract data from various sources, transform it into the desired format, and … load it into the appropriate data storage systems Collaborates with data scientists and analysts to optimize models and algorithms for data quality, security, and governance Integrates data from different sources, including databases, data warehouses, APIs, and external systems Ensures data consistency and integrity during the integration process, performing More ❯

Position: Analytics Engineer Location: London (hybrid 3 times a week)Department: ITType: 3 months rolling contractOutside IR35 The client is seeking an experienced Analytics Engineer to join its Data & Analytics team. The role focuses on building scalable data pipelines, transforming raw data into clean, reusable datasets, and enabling data-driven decision-making. … Key Responsibilities Design and build data products, with proficiency throughout the data lifecycle. Develop robust data models through close collaboration with business users and the engineering team. Partner with senior management, operational leads, and other stakeholders, coaching and supporting a data-driven culture, including KPI definition and reporting frameworks. Accountable for data extraction, transforming JSON & XML, utlising high experience within metadata management. Collaborate with data engineers to develop data, enrich product design and integrate data for predictive models or machine learning. Deliver well-defined, transformed, tested, documented, and code-reviewed datasets for analysis. Evaluate and recommend improvements in data flow, influencing and More ❯

Engineer Location: London (hybrid 3 times a week) Department: IT Type: 3 months rolling contract Outside IR35 The client is seeking an experienced Analytics Engineer to join its Data & Analytics team. The role focuses on building scalable data pipelines, transforming raw data into clean, reusable datasets, and enabling data-driven decision-making. … Key Responsibilities Design and build data products, with proficiency throughout the data lifecycle. Develop robust data models through close collaboration with business users and the engineering team. Partner with senior management, operational leads, and other stakeholders, coaching and supporting a data-driven culture, including KPI definition and reporting frameworks. Accountable for data extraction, transforming JSON & XML, utlising high experience within metadata management. Collaborate with data engineers to develop data, enrich product design and integrate data for predictive models or machine learning. Deliver well-defined, transformed, tested, documented, and code-reviewed datasets for analysis. Evaluate and recommend improvements in data flow, influencing and More ❯

Location - London, Bristol or Manchester (1 day a month onsite) Duration - 6 months Rate - £550 - £600pd (inside ir35) As a Data Engineer in the Cyber and Domains Protection Team you will: Work within an Agile team to support the development of dashboards and build automated reports to meet the needs of technical and non-technical users Work with … the data analyst and user researcher to update relevant data models to allow business intelligence data to meet the organisation's specific needs Develop business intelligence reports that can be automated, reused and shared with users directly Implement data flows to connect operational systems, data for analytics and business intelligence … for ETL and data cataloging Amazon Redshift or Athena for data warehousing and analytics Lambda for event-driven data processing. ETL/ELT pipeline development : experience in designing, building, and maintaining robust, automated data pipelines. You should be comfortable with both the theory and practical application of extracting, transforming, and loading More ❯

Location - London, Bristol or Manchester (1 day a month onsite) Duration - 6 months Rate - £550 - £600pd (inside ir35) As a Data Engineer in the Cyber and Domains Protection Team you will: Work within an Agile team to support the development of dashboards and build automated reports to meet the needs of technical and non-technical users Work with … the data analyst and user researcher to update relevant data models to allow business intelligence data to meet the organisation's specific needs Develop business intelligence reports that can be automated, reused and shared with users directly Implement data flows to connect operational systems, data for analytics and business intelligence … for ETL and data cataloging Amazon Redshift or Athena for data warehousing and analytics Lambda for event-driven data processing. ETL/ELT pipeline development : experience in designing, building, and maintaining robust, automated data pipelines. You should be comfortable with both the theory and practical application of extracting, transforming, and loading More ❯

Job Title: Manager - Cloud Data Architect (AWS & Snowflake) Protiviti's UK ED&A practice offers a comprehensive range of data use cases delivered through various delivery and commercial routes. We work on the full data lifecycle with highly skilled and experienced data professionals. Our solutions range from data strategy and … governance through the development, design and implementation of advanced analytics and digitisation. About the Role We are seeking a Manager-level Cloud Data Architect with deep expertise in AWS cloud services , Snowflake , and AWS-native data platforms to lead the design and implementation of scalable, secure, and high-performance data solutions. You will play … a pivotal role in shaping our cloud data strategy, driving innovation, and mentoring teams to deliver enterprise-grade data architectures that enable advanced analytics and business intelligence for our clients. Key Responsibilities Architecture & Design Design end-to-end cloud data architectures using AWS and Snowflake, ensuring scalability, security, and performance. Define and promote best More ❯

energy and utilities, financial services, government and public services, health and life sciences, and transport. The Data Engineer will have experience in AWS cloud technologies for ETL pipeline, data warehouse and data lake design/building and data movement. You will join the business at a period of huge growth. WE … growth area for the business with a diverse and growing capability, and we are looking for a Data Engineer with experience in AWS cloud technologies for ETL pipeline, data warehouse and data lake design/building and data movement. AWS data and analytics services (or open-source equivalent) such … requirements You thrive in problem-solving and analytical thinking You enjoy collaborating with multiple stakeholders in a fast-paced environment Experience in the design and deployment of production data pipelines from ingestion to consumption within a big data architecture, using Java, Python, Scala, Spark, SQL. Experience performing tasks such as writing scripts, extracting data More ❯

Data Architect - Contract We are seeking a skilled Data Architect to lead the design, development, and oversight of our enterprise-wide data infrastructure. This role plays a key part in defining our data strategy and ensuring high standards of data quality, accessibility, and security across multiple systems and platforms. The … ideal candidate will be a strategic thinker with strong expertise in modern data architectures and data governance practices. Required Skills and Experience: Proven experience as a Data Architect or in a similar senior data-focused role. Extensive experience with various data modelling techniques and tools. Deep understanding of different data … solutions (e.g., relational, NoSQL, data lakes) and their appropriate use cases. Strong knowledge of data integration patterns, ETL/ELT processes, and data pipeline orchestration. Experience with AWS data platforms and their respective data services. Solid understanding of data governance principles, including data quality, metadata More ❯

Data Architect - Contract We are seeking a skilled Data Architect to lead the design, development, and oversight of our enterprise-wide data infrastructure. This role plays a key part in defining our data strategy and ensuring high standards of data quality, accessibility, and security across multiple systems and platforms. The … ideal candidate will be a strategic thinker with strong expertise in modern data architectures and data governance practices. Required Skills and Experience: Proven experience as a Data Architect or in a similar senior data-focused role. Extensive experience with various data modelling techniques and tools. Deep understanding of different data … solutions (e.g., relational, NoSQL, data lakes) and their appropriate use cases. Strong knowledge of data integration patterns, ETL/ELT processes, and data pipeline orchestration. Experience with AWS data platforms and their respective data services. Solid understanding of data governance principles, including data quality, metadata More ❯

Senior Data Engineer (Azure) 2-3 days on-site - central London up to £75k + 20% Bonus + Excellent Beneifts Our client is a leading global Retail/FMCG brand undergoing an exciting period of rapid growth and transformation. With significant investment in data and technology, they are building a world-class data platform … across every area of the business - from supply chain and logistics to marketing, customer sales and in-store operations. As part of this journey, they are expanding their Data Engineering team and looking for a Senior Data Engineer (Azure) to help shape and scale the next generation of data products and pipelines. The Role … As a Senior Data Engineer, you'll play a key role in designing, building, and optimising scalable data pipelines and solutions on Microsoft Azure. You'll work closely with Data Architects, Analysts, and stakeholders in the business to expand the current data platform - which already powers sales and performance dashboards - into a More ❯

Principal Data Engineer - Core Systems Team Hybrid in London/Remote within UK The company Imagine a world where every small business has the power to thrive. That's the world we're building at iwoca. Small businesses aren't just statistics - they're the heartbeat of our communities, the character of our high streets, and the engine … leading fintech innovators. But we're just getting started. Our mission? To empower one million businesses with the financial tools they deserve. We combine cutting-edge technology and data science with genuine human understanding to make finance feel less like a barrier and more like a superpower. Whether it's managing cash flow or seizing unexpected opportunities, we … funds they need - often within minutes. The team You'll join the Core Systems team, who are responsible for driving innovation across the business by optimising development, building data systems, and continuously improving iwoca products. We follow Agile-inspired processes, using continuous integration and delivery, so that features go live in days or weeks, not months or years. More ❯

Data Engineer Active SC essential Croydon - up to 1 day per week £500/day outside until the end of June 2026 Please note, this in an outside IR-35 position on a Statement of Work basis, ideally we would prefer candidates with experience working to those arrangements. Essential skills: AWS Data Engineering Snowflake Databricks Responsibilities … Develop and manage data models, schemas, and metadata to support analytics and reporting needs. Collaborate with Data Analysts, Scientists, and Business Analysts to ensure data availability and usability. Implement data quality checks, validation routines, and monitoring to ensure data integrity and reliability. Optimise data workflows for performance, cost-efficiency, and maintainability using tools such as Azure Data Factory, AWS Data Pipeline for Data Orchestration, Databricks, or Apache Spark. Support the integration of data into visualisation platforms (e.g. Power BI, ServiceNow) and other analytical environments. Ensure compliance More ❯

Data Analytics Engineer Location: Central London - hybrid working Rate: £500.00 per day inside IR35 Duration: 12 months We are working with a leading global player in the travel industry who are seeking a Data Analytics Engineer to join their data team. This role sits at the intersection of data analysis and data engineeringideal for someone who enjoys building scalable data solutions while staying close to business impact. The Role As a Data Analytics Engineer , youll design, build, and maintain reliable data pipelinesprimarily using PySpark, SQL, and Python to ensure business teams (analysts, product managers, finance, operations) have access to well-modeled, actionable data. Youll … Engineers, youll be closer to the business, shaping analytics-ready datasets. Unlike pure Data Analysts, youll spend more time coding, managing data infrastructure, and ensuring pipeline reliability. Who Were Looking For Data Analytics : Analysts who have strong experience building and maintaining data pipelines (particularly in PySpark/SQL ) and want to More ❯

Data Engineer (with Data Analytics Background) Location: City of London Employment Type: Full-time Salary: £90,000-£100,000 Sector: Fintech We're looking for a well-rounded, communicative Data Engineer with a strong background in data analytics and experience within the Fintech sector . This role is ideal for someone who … began their career as a Data Analyst and has since transitioned into a more engineering-focused position, someone who enjoys understanding the business context just as much as building the data solutions behind it. You'll work extensively with Python , Snowflake , SQL , and dbt to design, build, and maintain scalable, high-quality data pipelines … and models that support decision-making across the business. This is a hands-on, collaborative role, suited to someone who's confident communicating with data, product, and engineering teams, not a 'heads-down coder' type. Top 4 Core Skills Python - workflow automation, data processing, and ETL/ELT development. Snowflake - scalable data architecture, performance More ❯

you, this means working on challenging projects that demand innovation, collaboration, and thought leadership . Role – We are looking for a talented Analytics Engineer to join our growing data and analytics team. You will play a pivotal role in shaping the organisation’s data-driven strategy, working across data engineering, analytics, and business domains … to design scalable, high-quality data models and pipelines. This is a fantastic opportunity to work on cutting-edge projects using technologies such as Snowflake, dbt, Data Vault, and Azure/AWS , while collaborating with data architects, data scientists, and business stakeholders to deliver insights that drive measurable impact. Key Responsibilities: Act … as the bridge between engineering teams and business stakeholders , translating requirements into data solutions. Design, build, and maintain data pipelines and transformations that turn raw data into structured, analysis-ready datasets. Develop and optimise data models (including Data Vault and dimensional modelling) to support large-scale analytics. Implement and enforce More ❯

you, this means working on challenging projects that demand innovation, collaboration, and thought leadership . Role – We are looking for a talented Analytics Engineer to join our growing data and analytics team. You will play a pivotal role in shaping the organisation’s data-driven strategy, working across data engineering, analytics, and business domains … to design scalable, high-quality data models and pipelines. This is a fantastic opportunity to work on cutting-edge projects using technologies such as Snowflake, dbt, Data Vault, and Azure/AWS , while collaborating with data architects, data scientists, and business stakeholders to deliver insights that drive measurable impact. Key Responsibilities: Act … as the bridge between engineering teams and business stakeholders , translating requirements into data solutions. Design, build, and maintain data pipelines and transformations that turn raw data into structured, analysis-ready datasets. Develop and optimise data models (including Data Vault and dimensional modelling) to support large-scale analytics. Implement and enforce More ❯