101 to 125 of 475 Data Pipeline Jobs in London

the best talent to strengthen our reputation as both a design practice and an employer of choice. We are seeking a motivated Junior BI Analyst to join our data analytics team. This role is perfect for an entry-level professional looking to build expertise in business intelligence, data management, and emerging AI technologies. The successful candidate … will support data-driven decision making across the organization while developing technical skills in data analysis and visualization. Key Responsibilities: Data Management & Quality Perform data cleansing, validation, and quality assurance processes Identify and resolve data inconsistencies and anomalies Document data sources, transformations, and cleansing procedures Monitor data pipelines and ensure timely data updates Business Intelligence & Reporting Develop and maintain standard business reports and dashboards Create ad-hoc reports based on stakeholder requirements Support the creation of KPIs and performance metrics Assist in data visualization best practices implementation AI Integration & Prompt Engineering Create and optimize AI prompts for various business applications Collaborate More ❯

Senior Data Engineer - (AI/ML Deployment Experience Required) Location: London (Hybrid: Monday-Wednesday in office) Salary: £65,000-£75,000 DoE Job Ref: J13026 An AI-first SaaS company on a mission to help organisations unlock the full potential of their first-party data is seeking a forward-thinking Senior Data Engineer with … strong deployment experience to help manage and scale its intelligent platform. Data sits at the heart of everything they do - transforming complex, messy datasets into actionable insights that empower smarter decisions and accelerate growth across industries. As they enter an exciting phase of rapid expansion, they're looking for a talented engineer to join them on this journey … and help shape the future of their data-driven platform. The Role The Senior Data Engineer will take ownership, shape the data foundations, and help power the next generation of AI-driven solutions. This is a hands-on, high-impact role where you'll design and build cloud-native data pipelines, champion More ❯

Senior Data Engineer – SC Cleared We are seeking a hands-on Senior Data Engineer with deep expertise in building and managing streaming and batch data pipelines . The ideal candidate will have strong experience working with large-scale data systems operating on cloud-based platforms such as AWS, Databricks or Snowflake . … This role also involves close collaboration with hyperscalers and data platform vendors to evaluate and document Proofs of Concept (PoCs) for modern data platforms, while effectively engaging with senior stakeholders across the organisation. Key Responsibilities: Design, develop, and maintain streaming and batch data pipelines using modern data engineering tools and frameworks. Work … with large volumes of structured and unstructured data , ensuring high performance and scalability. Collaborate with cloud providers and data platform vendors (e.g., AWS, Microsoft Azure, Databricks, IBM, Snowflake) to conduct PoCs for data platform solutions. Evaluate PoC outcomes and provide comprehensive documentation including architecture, performance benchmarks, and recommendations. Engage with key stakeholders including Heads More ❯

/Engineer Location: 3 Days p/w Central London Rate: £450 Outside IR35 Duration: 6 Months + Extensions This role is responsible for migrating and developing modern data and analytics solutions using Microsoft Fabric and Azure technologies. The position supports enterprise-wide BI initiatives, delivering data models, pipelines, dashboards and reporting assets that enable data-driven decision-making. Key Responsibilities Lead and implement data migration into Microsoft Fabric for analytics and reporting use cases Design and build data models from multiple sources to generate actionable insights Maintain and optimise data warehouse platforms; identify and resolve issues Develop data pipelines using Fabric Pipelines, Azure Data … Lakehouse and Warehouse environments Implement CI/CD processes using Azure DevOps for secure, reliable deployment Technical Skills: Strong expertise in: Power BI and paginated reporting SQL and data architecture Dimensional modelling (star schema, snowflake, denormalised structures, SCD handling) DAX, Visual Studio and data transformation logic Azure Fabric, Azure Data Factory, Synapse, Data More ❯

/Engineer Location: 3 Days p/w Central London Rate: £450 Outside IR35 Duration: 6 Months + Extensions This role is responsible for migrating and developing modern data and analytics solutions using Microsoft Fabric and Azure technologies. The position supports enterprise-wide BI initiatives, delivering data models, pipelines, dashboards and reporting assets that enable data-driven decision-making. Key Responsibilities Lead and implement data migration into Microsoft Fabric for analytics and reporting use cases Design and build data models from multiple sources to generate actionable insights Maintain and optimise data warehouse platforms; identify and resolve issues Develop data pipelines using Fabric Pipelines, Azure Data … Lakehouse and Warehouse environments Implement CI/CD processes using Azure DevOps for secure, reliable deployment Technical Skills: Strong expertise in: Power BI and paginated reporting SQL and data architecture Dimensional modelling (star schema, snowflake, denormalised structures, SCD handling) DAX, Visual Studio and data transformation logic Azure Fabric, Azure Data Factory, Synapse, Data More ❯

/Engineer Location: 3 Days p/w Central London Rate: £450 Outside IR35 Duration: 6 Months + Extensions This role is responsible for migrating and developing modern data and analytics solutions using Microsoft Fabric and Azure technologies. The position supports enterprise-wide BI initiatives, delivering data models, pipelines, dashboards and reporting assets that enable data-driven decision-making. Key Responsibilities Lead and implement data migration into Microsoft Fabric for analytics and reporting use cases Design and build data models from multiple sources to generate actionable insights Maintain and optimise data warehouse platforms; identify and resolve issues Develop data pipelines using Fabric Pipelines, Azure Data … Lakehouse and Warehouse environments Implement CI/CD processes using Azure DevOps for secure, reliable deployment Technical Skills: Strong expertise in: Power BI and paginated reporting SQL and data architecture Dimensional modelling (star schema, snowflake, denormalised structures, SCD handling) DAX, Visual Studio and data transformation logic Azure Fabric, Azure Data Factory, Synapse, Data More ❯

/Engineer Location: 3 Days p/w Central London Rate: £450 Outside IR35 Duration: 6 Months + Extensions This role is responsible for migrating and developing modern data and analytics solutions using Microsoft Fabric and Azure technologies. The position supports enterprise-wide BI initiatives, delivering data models, pipelines, dashboards and reporting assets that enable data-driven decision-making. Key Responsibilities Lead and implement data migration into Microsoft Fabric for analytics and reporting use cases Design and build data models from multiple sources to generate actionable insights Maintain and optimise data warehouse platforms; identify and resolve issues Develop data pipelines using Fabric Pipelines, Azure Data … Lakehouse and Warehouse environments Implement CI/CD processes using Azure DevOps for secure, reliable deployment Technical Skills: Strong expertise in: Power BI and paginated reporting SQL and data architecture Dimensional modelling (star schema, snowflake, denormalised structures, SCD handling) DAX, Visual Studio and data transformation logic Azure Fabric, Azure Data Factory, Synapse, Data More ❯

/Engineer Location: 3 Days p/w Central London Rate: £450 Outside IR35 Duration: 6 Months + Extensions This role is responsible for migrating and developing modern data and analytics solutions using Microsoft Fabric and Azure technologies. The position supports enterprise-wide BI initiatives, delivering data models, pipelines, dashboards and reporting assets that enable data-driven decision-making. Key Responsibilities Lead and implement data migration into Microsoft Fabric for analytics and reporting use cases Design and build data models from multiple sources to generate actionable insights Maintain and optimise data warehouse platforms; identify and resolve issues Develop data pipelines using Fabric Pipelines, Azure Data … Lakehouse and Warehouse environments Implement CI/CD processes using Azure DevOps for secure, reliable deployment Technical Skills: Strong expertise in: Power BI and paginated reporting SQL and data architecture Dimensional modelling (star schema, snowflake, denormalised structures, SCD handling) DAX, Visual Studio and data transformation logic Azure Fabric, Azure Data Factory, Synapse, Data More ❯

Data Delivery and Integration Manager - Full time permanent role - Retail industry - Remote/home-based - Salary up to £75,000 plus bonus, private healthcare and more! A large and growing retail business are looking to expand their IT function with the addition of a Data Integration Manager. This role will focus on end to end data engineering and integration solutions for a company wide modernisation programme. Duties will include: - Managing the processes for data engineering and data integration in order to align to business needs - Leading a team across engineering and integration - Work to ensure the data infrastructure of the business effectively supports analytics, AL, machine learning ML … and reporting - Designing of data pipelines, ETL, ELT processes and architecture - Work with internal stakeholders and multiple 3rd parties to seamlessly integrate systems, platforms and applications To be considered suitable for this Data Engineering Manager role you will need to have experience across the following: - Team leadership and guidance - Data engineering and aligning of More ❯

Data Engineer - II Would you like to ensure the successful delivery of the Data Platform and Software Innovations? Do you enjoy creating a collaborative and customer-focused working environment? About Team: LexisNexis Intellectual Property, which serves customers in more than 150 countries with 11,300 employees worldwide, is part of RELX, a global provider of information … based analytics and decision tools for professional and business customers. About Role: As a Data Engineer at LexisNexis Intellectual Property (LNIP), you'll contribute to building and maintaining our next-generation Strategic Data Platform. This platform ingests, enriches, and transforms global patent and IP-related data to power key products like PatentSight+, as well … as a growing ecosystem of internal tools and customer-facing solutions. In this early-career role, you will collaborate with senior engineers and technical leads to design robust data pipelines, apply engineering best practices, and support the delivery of high-quality data through modern platforms such as Databricks, APIs, and event-driven systems. You'll gain More ❯

Lead Data Engineer Salary/Rate : £100,000 - £120,000 per annum + Bonus Location : North London Company : Retelligence About Retelligence Retelligence is partnering with a high-growth, forward-thinking organization that specializes in digital innovation and marketing across international markets. The company is on an exciting journey, rapidly scaling its capabilities and leveraging advanced technology to deliver … cutting-edge solutions. Join a dynamic team within a business that values innovation, supports professional development, and offers exceptional career progression. The Role Retelligence is seeking a Lead Data Engineer to take a hands-on role in designing and delivering robust, real-time data pipelines and infrastructure in a Google Cloud Platform (GCP) environment. The company … Build and optimize data models for querying and analytics use cases. Develop fault-tolerant, highly available data ingestion and processing pipelines. Continuously monitor and improve pipeline performance for low-latency and high-throughput operations. Ensure data quality, integrity, and security across all systems. Implement effective monitoring, logging, and alerting mechanisms. Collaborate with product More ❯

Robert Half Technology are assisting a cutting edge AI organisation to recruit a Data Engineer on a contract basis - remote working - UK based A Junior Data Engineer who's excited to help build the data foundations that power cutting-edge AI solutions. You'll join a high-impact team working at the intersection of … data, analytics, and machine learning - designing pipelines and infrastructure that make innovation possible at scale. Role Design, build, and maintain scalable data pipelines that fuel AI and analytics initiatives. Partner closely with data scientists, analysts, and engineers to deliver clean, structured, and reliable data. Develop robust data transformations in Python and SQL … Engineer will ideally have 2-5 years of experience in data engineering or a similar role. Strong Python skills for data processing and pipeline development. Proven experience writing complex and efficient SQL. Hands-on Snowflake experience (data modelling, performance tuning, pipelines). Familiarity with orchestration tools (e.g., Airflow, dbt) is a More ❯

Use modern cloud-native tools and CI/CD platforms to maintain secure, observable, and high-performing environments. Relevant Skills: Essential Skills: Proficiency in Python or Scala for data … processing Strong SQL skills for querying and managing relational databases Experience with AWS services including S3, Glue, Redshift, Lambda, and Athena Knowledge of ETL processes and data pipeline development Understanding of data modelling and data warehousing concepts Familiarity with version control systems, particularly Git Desirable Skills: Experience with infrastructure as code tools such … as Terraform or CloudFormation Exposure to Apache Spark for distributed data processing Familiarity with workflow orchestration tools such as Airflow or AWS Step Functions Understanding of containerisation using Docker Experience with CI/CD pipelines and automated deployment processes Reasonable Adjustments: Respect and equality are core values to us. We are proud of the diverse and inclusive community More ❯

Use modern cloud-native tools and CI/CD platforms to maintain secure, observable, and high-performing environments. Relevant Skills: Essential Skills: Proficiency in Python or Scala for data … processing Strong SQL skills for querying and managing relational databases Experience with AWS services including S3, Glue, Redshift, Lambda, and Athena Knowledge of ETL processes and data pipeline development Understanding of data modelling and data warehousing concepts Familiarity with version control systems, particularly Git Desirable Skills: Experience with infrastructure as code tools such … as Terraform or CloudFormation Exposure to Apache Spark for distributed data processing Familiarity with workflow orchestration tools such as Airflow or AWS Step Functions Understanding of containerisation using Docker Experience with CI/CD pipelines and automated deployment processes Reasonable Adjustments: Respect and equality are core values to us. We are proud of the diverse and inclusive community More ❯

Data Delivery and Integration Manager (Remote) - Full time permanent role - Retail industry - Remote/home-based - Salary up to £75,000 plus bonus, private healthcare and more! A large and growing retail business are looking to expand their IT function with the addition of a Data Integration Manager. This role will focus on end to end … data engineering and integration solutions for a company wide modernisation programme. Duties will include: - Managing the processes for data engineering and data integration in order to align to business needs - Leading a team across engineering and integration - Work to ensure the data infrastructure of the business effectively supports analytics, AL, machine learning … ML and reporting - Designing of data pipelines, ETL, ELT processes and architecture - Work with internal stakeholders and multiple 3rd parties to seamlessly integrate systems, platforms and applications To be considered suitable for this Data Engineering Manager role you will need to have experience across the following: - Team leadership and guidance - Data engineering and aligning More ❯

Data Business Analyst - MetaData, Cloud, Healthcare £Market Rate - Inside IR35 London/Hybrid - 2 days per week on site 3-6 months My client is an instantly recognisable Healthcare Insurer who urgently require a Data Business Analyst to join a business-critical programme. Key Requirements: Proven commercial experience working as a Data Business Analyst … within large, complex organisations (Healthcare/Insurance preferred). Experience working on large-scale Data programmes. The ability to analyse and document business processes to gather and define requirements. The ability to develop and maintain detailed project documentation, including business requirements, process flows, and Data models. Working knowledge of structured, unstructured and MetaData. Familiarity with Data pipelines for cloud environments (Azure, GCP, AWS, Snowflake). Awareness of medallion architecture (Gold, Silver, Bronze). Previous experience creating Data requirements for Data Engineers/Data Scientists. Proficiency in Data Analysis tools such as SQL, Python and R and the ability to write simple SQL scripts. Strong understanding of More ❯

Senior Fabric Developer Data & Analytics Consultanc **y London/Hybri** d SDG Group is a global Consulting firm specialising in Data & Analytics. We are committed to unlocking an organisations' potential and hidden value by offering in-depth analytics expertise that empowers our clients' business models to become successful data-driven enterprises. Innovation is in … will play a pivotal role in designing, developing, and deploying enterprise-grade solutions using Microsoft Fabric. You will work closely with clients and internal teams to build scalable data pipelines, semantic models, and visualizations that support advanced analytics and business intelligence initiatives. Must be a willing to learn other Technology Stacks, Data Bricks, Sn **owflake. What … You'll** Be Doing Lead the design and implementation of solutions using Microsoft Fabric, including Data Factory, Dataflows, Lakehouses, and - Power BI.Architect and optimize data pipelines for performance, scalability, and re - liability.Develop and maintain semantic models and datasets for reporting and - analytics.Collaborate with cross-functional teams including data engineers, analysts, and business sta - keholders.Ensure More ❯

Company Description At The Stepstone Group, we have a simple yet very important mission: The right job for everyone. Using our data, platform, and technology, we create opportunities for job seekers and companies around the world to find a perfect match, in fair and equitable way. With over 20 brands across 30+ countries, we strive for fair and … the worlds leading job-tech platform. Job Description The role of Product Analytics Manager in our company is at the intersection of several disciplines: Product Analytics, Insights Analytics, Data Engineering and Data Science. Your teams projects will cover a broad range of topics, from working with engineers to implement event tracking on our platforms, all the … way to generating data insights and sometimes building predictive models, ultimately helping our Product organisation deliver great products, which solve meaningful user problems. You and your team are part of a larger Product Analytics group, and will be working closely with Product Directors, Heads of Product, Data Engineers, Data Scientists and many other stakeholders More ❯

LEAD DATA ENGINEER £80,000 - £90,000 - LONDON Company Our client is a global leader in data and analytics , working with major brands across 90+ markets. They combine deep industry expertise, vast datasets, and advanced technology to help businesses understand people and drive smarter decisions. The company is backed by a top-tier private equity firm … and is currently investing heavily in its data and technology platforms, offering the feel of a scale-up within a global business. Role As a Lead Data Engineer , you'll take ownership of designing, maintaining, and evolving the data pipelines that power large-scale analytical and machine learning models. You'll work closely with … Data Scientists, Analysts, and Software Engineers to deliver robust, scalable data infrastructure. The focus areas include building and optimising pipelines for survey data, improving anti-fraud models, and supporting new data science initiatives.You'll manage two mid-level engineers, providing technical direction, mentoring, and helping the team refine ways of working. The More ❯

Senior Data Engineer – Insurance Sector Location: Hybrid (London-based with flexible working) Salary: £85K -£100K – this is dependent on skills and experience Join a growing data team that’s transforming how the insurance industry uses data to make smarter, faster decisions. As a Senior Data Engineer , you’ll design and deliver scalable … high-performing data solutions that power analytics, reporting, and risk insights across the business. You’ll work closely with analysts, actuaries, and underwriters to build pipelines, models, and architectures that bring structure, speed, and accuracy to complex insurance data. This is a fantastic opportunity to make a real impact while shaping the future of data engineering … within a forward-thinking organisation. What You’ll Be Doing Partnering with analysts and product owners to turn business needs into scalable data solutions. Designing and maintaining robust data pipelines and models for analytics and reporting. Applying best practices in data architecture, security, and performance. Working with modern tools including Azure, Databricks, Python, Azure More ❯

Senior Data Engineer – Insurance Sector Location: Hybrid (London-based with flexible working) Salary: £85K -£100K – this is dependent on skills and experience Join a growing data team that’s transforming how the insurance industry uses data to make smarter, faster decisions. As a Senior Data Engineer , you’ll design and deliver scalable … high-performing data solutions that power analytics, reporting, and risk insights across the business. You’ll work closely with analysts, actuaries, and underwriters to build pipelines, models, and architectures that bring structure, speed, and accuracy to complex insurance data. This is a fantastic opportunity to make a real impact while shaping the future of data engineering … within a forward-thinking organisation. What You’ll Be Doing Partnering with analysts and product owners to turn business needs into scalable data solutions. Designing and maintaining robust data pipelines and models for analytics and reporting. Applying best practices in data architecture, security, and performance. Working with modern tools including Azure, Databricks, Python, Azure More ❯

Senior Data Engineer – Insurance Sector Location: Hybrid (London-based with flexible working) Salary: £85K -£100K – this is dependent on skills and experience Join a growing data team that’s transforming how the insurance industry uses data to make smarter, faster decisions. As a Senior Data Engineer , you’ll design and deliver scalable … high-performing data solutions that power analytics, reporting, and risk insights across the business. You’ll work closely with analysts, actuaries, and underwriters to build pipelines, models, and architectures that bring structure, speed, and accuracy to complex insurance data. This is a fantastic opportunity to make a real impact while shaping the future of data engineering … within a forward-thinking organisation. What You’ll Be Doing Partnering with analysts and product owners to turn business needs into scalable data solutions. Designing and maintaining robust data pipelines and models for analytics and reporting. Applying best practices in data architecture, security, and performance. Working with modern tools including Azure, Databricks, Python, Azure More ❯

Senior Data Engineer – Insurance Sector Location: Hybrid (London-based with flexible working) Salary: £85K -£100K – this is dependent on skills and experience Join a growing data team that’s transforming how the insurance industry uses data to make smarter, faster decisions. As a Senior Data Engineer , you’ll design and deliver scalable … high-performing data solutions that power analytics, reporting, and risk insights across the business. You’ll work closely with analysts, actuaries, and underwriters to build pipelines, models, and architectures that bring structure, speed, and accuracy to complex insurance data. This is a fantastic opportunity to make a real impact while shaping the future of data engineering … within a forward-thinking organisation. What You’ll Be Doing Partnering with analysts and product owners to turn business needs into scalable data solutions. Designing and maintaining robust data pipelines and models for analytics and reporting. Applying best practices in data architecture, security, and performance. Working with modern tools including Azure, Databricks, Python, Azure More ❯

has big plans for its team and the wider business. I am seeking an exceptional Engineering Manager within Analytics to lead high-performing teams at the crossroads of data engineering, analytics, and real-world business impact. This role offers the chance to shape the architecture, culture, and strategy of a modern analytics ecosystem , driving innovation, scalability, and measurable … different types of projects at one time and adaptable in leading teams and upskilling those around you. In this role, you will: Lead, mentor, and inspire analytics and data engineering teams, fostering technical excellence and a culture of innovation. Architect and optimise ELT pipelines and workflows using dbt, Airflow, SQL, and Python. Oversee the design and evolution of … are able to bring: Bring 5+ years of experience in analytics and data engineering, including proven leadership within technical teams. Demonstrate deep expertise in data pipeline architecture, schema design, and transformation frameworks. Have strong proficiency across cloud-based data platforms and scalable infrastructure. Excel at developing teams, driving process optimisation, and championing collaboration More ❯

has big plans for its team and the wider business. I am seeking an exceptional Engineering Manager within Analytics to lead high-performing teams at the crossroads of data engineering, analytics, and real-world business impact. This role offers the chance to shape the architecture, culture, and strategy of a modern analytics ecosystem , driving innovation, scalability, and measurable … different types of projects at one time and adaptable in leading teams and upskilling those around you. In this role, you will: Lead, mentor, and inspire analytics and data engineering teams, fostering technical excellence and a culture of innovation. Architect and optimise ELT pipelines and workflows using dbt, Airflow, SQL, and Python. Oversee the design and evolution of … are able to bring: Bring 5+ years of experience in analytics and data engineering, including proven leadership within technical teams. Demonstrate deep expertise in data pipeline architecture, schema design, and transformation frameworks. Have strong proficiency across cloud-based data platforms and scalable infrastructure. Excel at developing teams, driving process optimisation, and championing collaboration More ❯