26 to 50 of 492 Data Pipeline Jobs in the UK

Job summary The UKHSA Digital and Data Directorate provides strategic leadership and oversight for developing and managing a common and bespoke suite of performant, modern, secure, data and analytical tools, services, platforms and applications, ensuring access to high quality data and analytics services, internally and externally. Based within the Digital and Data Directorate, the Data Product Development Division are responsible for the development and delivery of new and existing dashboards and platforms, ensuring that data are made available to improve health security outcomes. Our flagship product is the UKHSA data dashboard, which builds on the hugely influential COVID-19 dashboard, and presents data on COVID-19, as well as other respiratory viruses such as influenza. Main duties of the job Collation, analysis, interpretation and visualisation of public health information. Implement data extraction, transformation, and loading (ETL) processes to ensure data quality and consistency. Building and maintaining interactive dashboards and data visualisation tools to facilitate data More ❯

Fabric tools to support innovative solutions. Collaborating with product owners and analysts, this role will ensure data flows and outputs drive CRB decision making. The role combines pipeline development and quality with dashboard delivery and results. Responsibilities The Role: Pipeline Development & Quality (≈ 60%) – Building Robust Data Infrastructure: • Build and maintain CRB specific data … dashboard performance, incorporating user feedback for improvements Qualifications The Requirements: • Bachelor's degree in Computer Science, Data Science, or related field • Experience in data engineering, pipeline development, or ETL processes • Proficiency in scripting (Python, SQL) and data integration tools • Expertise with Microsoft Azure/Fabric for data storage and processing • Knowledge … with APIs and data architecture in cloud environments • Ability to work in agile settings, collaborating with cross-functional teams • Strong problem-solving skills for data pipeline and quality issues •Proven delivery on data-driven projects •Ability to translate technical outputs into business-relevant results Enjoy a benefits package designed to help you thrive More ❯

Data Architect (Azure) 6 Month Contract UK Based Via Umbrella Our UK leading life and pensions client are looking for a Data Architect to joun their team on an initial 6 month contract. Key Responsibilities Data Architecture & Design Define and maintain enterprise data architecture standards, models, and frameworks. Design data solutions leveraging Azure services such as Azure Data Lake, Azure SQL Database, Azure Synapse Analytics, Azure Data Factory, and Azure Databricks. Data Integration & ETL Develop and optimize data pipelines for ingestion, transformation, and storage using Azure Data Factory and Databricks. Governance & Security Implement data governance, security … and compliance practices aligned with financial services regulations (e.g., GDPR, PCI DSS). Performance & Scalability Ensure data solutions are optimized for performance and scalability across large datasets. Collaboration Work closely with data engineers, analysts, and business stakeholders to deliver robust data platforms. Innovation Stay current with emerging Azure technologies and best practices in data More ❯

Lead Data Engineer Lead Data Engineer required to join my Clients Technology Team and contribute to their Group Architecture Board in building the next generation of their eCommerce data platform. You will play a pivotal role in designing and maintaining a modern, cloud-based data architecture. Working closely with product, technology, and … wider business stakeholders, you will be crucial in developing scalable data pipelines, improving reliability, and enabling smarter, data-driven insights and decisions. You will have a direct voice in how we design, build, and evolve our platform and the technologies we adopt! We are looking for a pragmatic, innovative, and self-motivated problem solver who delivers … Architecture Specialist: Design and develop robust data models, ETL/ELT processes, and data marts that support complex analytics and operational use cases. Pipeline Optimization: Maintain and optimize data pipelines and warehouse environments to ensure top-tier performance, reliability, and scalability. Cross-Functional Collaboration: Effectively collaborate with cross-functional teams to More ❯

Lead Data Engineer Lead Data Engineer required to join my Client’s Technology Team and contribute to their Group Architecture Board in building the next generation of their eCommerce data platform. You will play a pivotal role in designing and maintaining a modern, cloud-based data architecture. Working closely with product, technology … and wider business stakeholders, you will be crucial in developing scalable data pipelines, improving reliability, and enabling smarter, data-driven insights and decisions. You will have a direct voice in how we design, build, and evolve our platform and the technologies we adopt! We are looking for a pragmatic, innovative, and self-motivated problem solver who … Architecture Specialist: Design and develop robust data models, ETL/ELT processes, and data marts that support complex analytics and operational use cases. Pipeline Optimization: Maintain and optimize data pipelines and warehouse environments to ensure top-tier performance, reliability, and scalability. Cross-Functional Collaboration: Effectively collaborate with cross-functional teams to More ❯

algorithms. Lead the selection, validation, and optimization of models to discover meaningful patterns and insights, ensuring models remain relevant, reliable, and scalable. Drive continuous integration and deployment of data science solutions, optimizing performance through advanced machine learning techniques, code reviews, and best practices. 'Develop and deliver sophisticated visualizations, dashboards, and reports translate complex data into clear … actionable insights for business stakeholders. Present technical solutions to business stakeholders, using creative methods to explain complex concepts, increase understanding, and encourage solution adoption. Mentor and develop junior data scientists, fostering a culture of continuous learning, knowledge sharing, and skills development within the organization. Write clean, high-quality code, ensuring all outputs pass quality assurance checks, and contribute … Experience with MLOps practices including model drift detection, decay, A/B testing, integration testing, differential testing, Python package building, and code version control. Skilled in data pipeline creation and working with both structured and unstructured data. Familiar with cloud platforms (AWS, Azure, GCP) and containerization technologies like Docker and Kubernetes. Excellent problem-solving skills, combined with More ❯

Job Description: Data Engineer (with Tableau Specialism) Responsibilities: * Design, develop, and manage data models, schemas, and metadata to support advanced analytics and reporting requirements. * Work closely with Data Analysts, Data Scientists, and Business Analysts to ensure data availability, accessibility, and usability across the organisation. * Implement data quality checks … integrity and reliability. * Optimise data workflows for performance, cost-efficiency, and maintainability using tools such as Azure Data Factory, AWS Data Pipeline, Databricks, or Apache Spark . * Integrate and prepare data for Tableau dashboards and reports , ensuring optimal performance and alignment with business needs. * Collaborate with visualisation teams to … develop, maintain, and enhance Tableau data sources (including Extracts, Hyper files, and live connections) and to support self-service analytics. * Ensure compliance with data governance, security, and privacy policies , maintaining adherence to organisational standards. * Document data architecture, pipelines, and Tableau data models to promote transparency, maintainability, and knowledge sharing. * Contribute to the More ❯

the next couple of years. Join us - and help us fulfil our mission to close the world's digital skills gap. Key Skills: Architectural Comprehension & Alignment: Proactively understand data architectural designs for data services, applications, and persistence layers, ensuring all data models and schemas align with the overall architectural vision, technology choices, and data … architecture. Data Service Support: Contribute to the design of data persistence layers and define data schemas for API documentation (e.g., OpenAPI specifications). Pipeline Code Integrity: Experience using SOLID principles when designing and reviewing code for data transformation pipelines and code-based data models ensuring high code quality, testability … and stakeholder engagement skills Desirable Skills: Experience with API schema design and OpenAPI specifications Working knowledge of software development principles (e.g., SOLID, DRY) as applied to data pipeline code and modelling tools Why join AND Digital? We have three values: wonder, share, and delight. These values inform how we work with clients, and our culture: what it More ❯

Adword Job Title: Data Architect (Insurance Domain) Location: London UK Hybrid role Responsibilities: Experience Level 15 years with at least 5 years in Azure ecosystem Role: We are seeking a seasoned Data Architect to lead the design and implementation of scalable data solutions for a strategic insurance project The ideal candidate will have deep … expertise in Azure cloud services Azure Data Factory and Databricks with a strong understanding of data modeling data integration and analytics in the insurance domain Key Responsibilities Architect and design end to end data solutions on Azure for insurance related data workflows Lead data ingestion transformation and orchestration … using ADF and Databricks Collaborate with business stakeholders to understand data requirements and translate them into technical solutions Ensure data quality governance and security compliance across the data lifecycle Optimize performance and cost efficiency of data pipelines and storage Provide technical leadership and mentoring to data engineers and developers Mandatory More ❯

Adword Job Title: Data Architect (Insurance Domain) Location: London UK Hybrid role Responsibilities: Experience Level 15 years with at least 5 years in Azure ecosystem Role: We are seeking a seasoned Data Architect to lead the design and implementation of scalable data solutions for a strategic insurance project The ideal candidate will have deep … expertise in Azure cloud services Azure Data Factory and Databricks with a strong understanding of data modeling data integration and analytics in the insurance domain Key Responsibilities Architect and design end to end data solutions on Azure for insurance related data workflows Lead data ingestion transformation and orchestration … using ADF and Databricks Collaborate with business stakeholders to understand data requirements and translate them into technical solutions Ensure data quality governance and security compliance across the data lifecycle Optimize performance and cost efficiency of data pipelines and storage Provide technical leadership and mentoring to data engineers and developers Mandatory More ❯

Data Engineer Permanent, Full-time UK - Location TBC About Us James Fisher is a global engineering services company. From our origins as a ship owner and operator, we've evolved to provide the expertise and innovative technology our customers need in the harshest of environments across Energy, Defence and Maritime Transport. We are relied on to help our … leading provider of global maritime shipping and ship-to-ship (STS) transfer services. Together, we are building a brighter, bolder and more innovative future. The Role As a Data Engineer within Group IT, you will play a pivotal role in transforming raw data into meaningful insights that drive strategic decision-making and operational efficiency. You'll … be at the forefront of designing, implementing, and maintaining robust data systems that support key business objectives. This role offers the opportunity to contribute to high-impact strategic projects, collaborate with cross-functional teams across the organisation, and help shape the future of data engineering practices within the business. Principle Accountabilities Design, develop, and maintain scalable More ❯

Contract Data Engineer 3 month initial contract (with good chance of extension) Manchester, 2 days per week on-site Outside IR35 Start before Christmas Overview My client are a well knwon organsation just outside Manchester looking for a contract Data Engineer. You will review and improve a live Google Cloud data platform. You will … lead ingestion, transformation, quality, and reporting enablement. You will work with data colleagues on shared standards and delivery. Key Responsibilities Review the central data platform architecture, pipelines, and code standards Improve scalability, performance, monitoring, and disaster recovery Audit all data sources across the business Build secure ingestion pipelines for new sources Design and build … and an entity diagram Core Skills Strong hands-on Data Engineering experience Advanced Python for data transformation Google Cloud Platform production experience Data pipeline design and optimisation Data quality automation and monitoring Customer data and identity matching GDPR and data governance delivery Stakeholder communication in agile teams More ❯

- someone who gets excited about finding edge cases, building robust pipelines, and seeing their clean, validated datasets power accurate ML predictions. What You'll Do Data Pipeline Engineering Build production ETL pipelines that process mobile analytics events from AppsFlyer, Adjust, and other MMPs Engineer features for ML models: temporal patterns, user behaviour sequences, campaign attribution Ensure … real-world data: duplicates, out-of-order events, schema drift, missing fields, null handling Infrastructure & Orchestration Build Prefect workflows to orchestrate our multi-stage data pipeline Optimize and monitor BigQuery operations for cost and performance Establish CI/CD practices as we scale from a dozen clients to 100+ within 12 months Platform Development Build … pre-commit) Experience with messy real-world data: you've dealt with duplicates, nulls, schema drift, out-of-order events, and know how to build defensive pipelines Data quality mindset: you instinctively add validation checks, sanity-check implausible results, and think about edge cases before they become production bugs Right to Work in the UK (we More ❯

Data Engineer Location: Sheffield (Hybrid - 3 days per week onsite) Salary: £50,000-£60,000 depending on experience DCS Tech are searching for an experienced Data Engineer to join our clients growing team! You will play a crucial part in designing, building, and optimising the data infrastructure that underpins the organisation. Key responsibilities Design … develop, and deploy scalable, secure, and reliable data pipelines using modern cloud and data engineering tools. Consolidate data from internal systems, APIs, and third-party sources into a unified data warehouse or data lake environment. Build and maintain robust data models to ensure accuracy, consistency, and accessibility across … experience as a Data Engineer or in a similar data-centric role. Strong proficiency in SQL and Python. Solid understanding of ETL/ELT pipelines, data modelling, and data warehousing principles. Experience working with cloud platforms such as AWS, Azure, or GCP. Exposure to modern data tools such as Snowflake More ❯

Databricks Data Architectx2 UK Wide Hybrid Working £80,000-£90,000 As a Solution Architect with an Azure and Databrick focus, you will be an integral part of our team dedicated to building scalable and secure data platforms. You will leverage your expertise in Databricks, Apache Spark, and Azure to design, develop, and implement data warehouses, data lakehouses, and AI/ML models that fuel our data-driven operations. Duties Design and build high-performance data platforms: Utilize Databricks and Apache Spark to extract, transform, and load data into Azure Data Lake Storage and other Azure services. Design and oversee the delivery of … secure data warehouses and data lakehouses: Implement data models, data quality checks, and governance practices to ensure reliable and accurate data. Abilty to Design, Build and deploy AI/ML models: Integrate Machine Learning into data pipelines, leverage Databricks ML and Azure ML to develop predictive models and drive business More ❯

hunt for builders . No, we've not ventured into construction in our quest to conquer the world, rather a designer and builder of systems for all things data related where we are conquering the Data World. We can offer an interesting insight into projects spanning a variety of sectors, which may include industries such as … to explore our dynamic team where you can truly act as an expert in your field in support of our clients and their challenges in the world of data and technology, read on The Team We're Intuita - a fast growing consultancy that's making waves in both the consultancy and technology space. With our ambitious goals for … in the data pipelines. • Stay updated with the latest Azure technologies and industry best practices to continuously enhance data engineering capabilities. Looking at our current pipeline of work, we can also consider those with an Analytics Engineering lean, with experience within BigQuery (GCP) with data modelling in DBT and mobile/telecoms industry More ❯

Data Architect - Fully Remote role Data Architect sought by a well-established Data & Analytics consultancy. Our client is seeking a cloud professional to spearhead strategic leadership and technical expertise within their data solutions landscape, focussed on cloud data platforms, but including IoT analytics, data integration, and migration projects. … This opportunity is offered on a remote-first basis, but with an open invitation to their headquarters any time. Role Responsibilities: The Data Architect will be pivotal in shaping the approach to designing and developing data solutions using cloud technologies such as AWS and Azure. This position involves: Leading the design and implementation of innovative cloud … using Azure, AWS, or exceptional on-premise design expertise. At least 5 years in data engineering or business intelligence roles. Proficiency in ETL and data pipeline design, with a technology-agnostic approach. A solid understanding of data warehouse and data lake principles. Expert SQL skills and demonstrable data modelling More ❯

Data Architect - Fully Remote role Data Architect sought by a well-established Data & Analytics consultancy. Our client is seeking a cloud professional to spearhead strategic leadership and technical expertise within their data solutions landscape, focussed on cloud data platforms, but including IoT analytics, data integration, and migration projects. … This opportunity is offered on a remote-first basis, but with an open invitation to their headquarters any time. Role Responsibilities: The Data Architect will be pivotal in shaping the approach to designing and developing data solutions using cloud technologies such as AWS and Azure. This position involves: Leading the design and implementation of innovative cloud … using Azure, AWS, or exceptional on-premise design expertise. At least 5 years in data engineering or business intelligence roles. Proficiency in ETL and data pipeline design, with a technology-agnostic approach. A solid understanding of data warehouse and data lake principles. Expert SQL skills and demonstrable data modelling More ❯

We're looking for an experienced data engineer who specialises in designing, constructing, and maintaining data pipelines using Azure Data Factory (ADF) and Databricks. In this role, you will collaborate closely with data scientists, business analysts, and other stakeholders to meet the organisation's data requirements. The role is open … and dynamic work environment. Comprehensive Benefits: Enjoy benefits like remote working in Ireland or the UK, health insurance, flexible working hours & company trips. Responsibilities: Designing, constructing, and maintaining data pipelines using ADF and Databricks. Collaborating with data scientists, business analysts, and stakeholders to understand data needs and provide effective data solutions. Developing … and managing scalable data architectures, ensuring data quality and integrity. Optimising data pipelines for performance and reliability. Monitoring and troubleshooting data pipelines to ensure smooth operations. Implementing data security and privacy measures to safeguard sensitive information. Keeping up-to-date with emerging technologies and industry trends in data More ❯

Role Title: Data Engineer Role Type: Contract (inside ir35) Role Location: London, UK Mandatory: Active Security Clearance Job Description:- Essential Skills & Experience: ·10+ years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks. ·Good proficiency in Python and Spark (PySpark) or Scala. ·Deep understanding of data warehousing principles, data modelling techniques, and data integration patterns. ·Extensive experience with Azure data services, including Azure Data Factory … Azure Blob Storage, and Azure SQL Database. ·Experience working with large datasets and complex data pipelines. ·Experience with data architecture design and data pipeline optimization. ·Proven expertise with Databricks, including hands-on implementation experience and certifications. ·Experience with SQL and NoSQL databases. ·Experience with data quality and data governance More ❯

Job Title: Data Engineer Location: Hybrid Contract Type: Permanent/Full-Time About the Role Are you passionate about technology and eager to make a real impact? At Buzz Bingo, we’re looking for a Data Engineer who thrives on innovation and enjoys working across a diverse technology stack. The systems you’ll support underpin both … in-club and online customer experiences, giving you the opportunity to influence how thousands of people interact with Buzz Bingo every day. What You’ll Do Data Pipeline Development: Design, implement, and maintain robust ETL/ELT pipelines for ingesting and transforming data from multiple sources. Data Modelling: Create and maintain models that … data solutions. Data Quality Assurance: Implement checks and monitoring processes to ensure accuracy and reliability. Documentation: Maintain detailed technical documentation for data architectures, pipeline designs, and operational procedures. Performance Tuning: Analyse and optimize workflows for performance and cost efficiency. Innovation: Stay current with emerging technologies and best practices to continuously improve our data More ❯

Data Engineer (with Data Analytics Background) Location: City of London Employment Type: Full-time Salary: £90,000-£100,000 Sector: Fintech We're looking for a well-rounded, communicative Data Engineer with a strong background in data analytics and experience within the Fintech sector . This role is ideal for someone who … began their career as a Data Analyst and has since transitioned into a more engineering-focused position, someone who enjoys understanding the business context just as much as building the data solutions behind it. You'll work extensively with Python , Snowflake , SQL , and dbt to design, build, and maintain scalable, high-quality data pipelines … and models that support decision-making across the business. This is a hands-on, collaborative role, suited to someone who's confident communicating with data, product, and engineering teams, not a 'heads-down coder' type. Top 4 Core Skills Python - workflow automation, data processing, and ETL/ELT development. Snowflake - scalable data architecture, performance More ❯

Senior Databricks Data Engineer 2-3 days on-site - central London up to £75k + 20% Bonus + Excellent Benefits Our client is a leading global Retail/FMCG brand undergoing an exciting period of rapid growth and transformation. With significant investment in data and technology, they are building a world-class data platform … power decision-making across every area of the business - from supply chain and logistics to marketing, customer sales and in-store operations. We are seeking an experienced Senior Data Engineer with deep expertise in Databricks and Medallion Architecture (Bronze-Silver-Gold) to design, build, and optimize our next-generation data platform. This role will be pivotal … in developing scalable data pipelines, enabling advanced analytics, and driving data quality and governance across the organization. You'll work closely with data scientists, analysts, and business stakeholders to transform raw data into trusted, actionable insights that power critical business decisions. Key Responsibilities Design and implement scalable data pipelines and More ❯

We are looking for a Lead Data Architect to provide strategic leadership and technical expertise in the design and development of data solutions. This role involves setting technical standards, evaluating new technologies, and shaping our approach to data. You will lead a team of Data Architects, Engineers, and Analysts, defining data architecture … frameworks and ensuring the development of scalable, robust, and cost-effective solutions. The ideal candidate will have a passion for data, be technology-agnostic, and possess excellent client-facing skills. Lead Data Architect Annual Salary: circa £85,000 Location: Flexible, hybrid working with some client travel Day to day of the role: Shape the approach to … platforms in Azure/AWS or significant on-premise design experience. 5+ years in data engineering or business intelligence roles. Extensive ETL and data pipeline design experience, technology agnostic. Proficiency in SQL and experience with data engineering coding languages such as Python, R, or Spark. Understanding of data warehouse and More ❯

Robert Half have partnered with a market leading property development organisation for an experienced Azure Data Engineer to join their data and analytics team, on an initial 3-6 month contract based out of their London office The ideal candidate will have a strong background in designing, building, and maintaining scalable data pipelines in … the Azure cloud environment, with hands-on experience in modern data platforms and ETL frameworks. Key Responsibilities: Design, develop, and maintain data pipelines and data integration solutions using Azure Data Factory, Synapse Analytics, and related services. Build and optimise data models to support reporting, analytics, and machine learning use cases. … Implement robust data quality, governance, and security practices across the data ecosystem. Collaborate with analysts, data scientists, and business stakeholders to understand data needs and translate them into technical solutions. Work with structured and unstructured data sources, ensuring scalability, reliability, and performance. Monitor and troubleshoot data workflows, ensuring More ❯