1 to 25 of 1,392 Data Pipeline Jobs in the UK

that solves real-world problems for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on … re underpinned by over 300 engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data architectures such as Data … data engineering projects. You will work closely with cross-functional teams and contribute to the strategic direction of our data initiatives. RESPONSIBILITIES Data Pipeline Development: Lead the design, implement, and maintain scalable data pipelines for ingesting, processing, and transforming large volumes of data from various sources using tools such More ❯

a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data … technical solutions. Communicate technical concepts effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote and implement DevOps best practices. … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data … technical solutions. Communicate technical concepts effectively to both technical and non-technical audiences. Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. Promote and implement DevOps best practices. … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

that solves real-world problems for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on … re underpinned by over 300 engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data architectures such as Data … modelling, ETL (Extract, Transform, Load) processes, and big data technologies it becomes possible to develop robust and reliable data solutions. RESPONSIBILITIES Data Pipeline Development: Design, implement, and maintain scalable data pipelines for ingesting, processing, and transforming large volumes of data from various sources using tools such as databricks More ❯

Senior Data Engineer Salary: around £75k - plus a very attractive on top Location: London or Leeds - relaxed about hybrid working, if preferred About Us Sports forecasting company specializing in developing player level, play-by-play simulators that generate highly accurate, near-instant outcome projections for sporting events. Since its establishment, the company has focused on serving the emerging … grown into a globally recognized leader in sports modelling and analytics. Purpose of role: The purpose of this role is to design, build, and maintain scalable and efficient data transformation workflows that empower the business with actionable insights and impactful visualisations. This role bridges the gap between data engineering and business intelligence by owning the transformation … layer and enabling clear, trusted, and timely analytics across the organisation. The successful candidate will have a strong grasp of modern data modelling practices, analytics tooling, and interactive dashboard development in Power BI and Plotly/Dash. Key responsibilities: Designing and maintaining robust data transformation pipelines (ELT) using SQL, Apache Airflow, or similar tools. Building and More ❯

Senior Data Engineer Salary: around £75k - plus a very attractive on top Location: London or Leeds - relaxed about hybrid working, if preferred About Us Sports forecasting company specializing in developing player level, play-by-play simulators that generate highly accurate, near-instant outcome projections for sporting events. Since its establishment, the company has focused on serving the emerging … grown into a globally recognized leader in sports modelling and analytics. Purpose of role: The purpose of this role is to design, build, and maintain scalable and efficient data transformation workflows that empower the business with actionable insights and impactful visualisations. This role bridges the gap between data engineering and business intelligence by owning the transformation … layer and enabling clear, trusted, and timely analytics across the organisation. The successful candidate will have a strong grasp of modern data modelling practices, analytics tooling, and interactive dashboard development in Power BI and Plotly/Dash. Key responsibilities: Designing and maintaining robust data transformation pipelines (ELT) using SQL, Apache Airflow, or similar tools. Building and More ❯

over 10,000 people. Its big-scale stuff and we’re still growing. Job Purpose With a big investment into Databricks, and with a large amount of interesting data, this is the chance for you to come and be part of an exciting transformation in the way we store, analyse and use data in a fast … paced organisation. You will join as a Senior Platform Data Engineer providing technical leadership to the Data Engineering team. You will work closely with our Data Scientists and business stakeholders to ensure value is delivered through our solutions. Job Accountabilities Develop robust, scalable data pipelines to serve the easyJet analyst and data … time in a commercial production environment with Spark Streaming, Kafka, DLT or Beam. Understanding of the challenges faced in the design and development of a streaming data pipeline and the different options for processing unbounded data (pubsub, message queues, event streaming etc) Understanding of the most commonly used Data Science and Machine Learning More ❯

Senior Data Engineer – Assistant Manager Base Location: Leeds based (Hybrid – 3 days per week in office) As a result of the work that we do, we require applicants to hold or be capable of obtaining UK National Security Vetting, the requirements for which could include but not be limited to having resided in the UK for at least … for the future of the planet or everything in-between. In a world where tech is always changing, so are we. Why Join KPMG Curve as a Senior Data Engineer Progression at pace. Innovation, led by constant learning. Work that excites you with every twist and turn. It all starts at KPMG Curve. Here, you’ll progress your … and experience using at least one Data Platform Technology such as Quantexa, Palantir and DataBricks Knowledge of test automation frameworks and ability to automate testing within the pipeline To discuss this or wider Technology roles with our recruitment team, all you need to do is apply, create a profile, upload your CV and begin to make your More ❯

to technology, delivering bespoke software solutions and expert advice. Our clients are increasingly looking to us to help them make the best use of their data. In building data platforms and pipelines, our data engineers create the foundation for diverse data & analytics solutions, including data science and AI. They build data lakes and warehouses, create the processes to extract or access operational data, and transform siloed datasets into integrated data models that allow insight into business performance and problems or training of ML models. These are hands-on, client-facing roles, with openings at senior or lead level to suit your experience. You may be … or solving tough engineering challenges. You'd also be expected to spend some time on-site with clients in the London area on an ad-hoc basis. Our data engineers combine a strong software engineering approach with solid data fundamentals and experience with modern tools. We're technology agnostic, and we're open minded when it More ❯

Be relevant - Relevant to our people, our partners and the planet Are you ready to join the Fearless? Job Description We are currently looking for a Mid-Senior Data Engineer to join us in our growing Data Engineering team; to help develop, maintain, support, and integrate our growing number of data systems. You will … be instrumental in designing, building, and maintaining robust and scalable data pipelines that power our operational subscribers and analytical platforms. You will work with a diverse range of data sources, integrating both real-time streams and micro-batches, connecting with varied end points to move data at speed and at scale. The right candidate … to design and implement logical and physical data models for our analytical and operational data stores. Monitoring and Performance Tuning: Proactively monitor data pipeline performance, identify bottlenecks, and implement optimizations to ensure low latency and high throughput. Collaboration and Mentorship: Collaborate with cross-functional teams, including software engineers, data scientists, and More ❯

Be relevant - Relevant to our people, our partners and the planet Are you ready to join the Fearless? Job Description We are currently looking for a Mid-Senior Data Engineer to join us in our growing Data Engineering team; to help develop, maintain, support, and integrate our growing number of data systems. You will … be instrumental in designing, building, and maintaining robust and scalable data pipelines that power our operational subscribers and analytical platforms. You will work with a diverse range of data sources, integrating both real-time streams and micro-batches, connecting with varied end points to move data at speed and at scale. The right candidate … to design and implement logical and physical data models for our analytical and operational data stores. Monitoring and Performance Tuning: Proactively monitor data pipeline performance, identify bottlenecks, and implement optimizations to ensure low latency and high throughput. Collaboration and Mentorship: Collaborate with cross-functional teams, including software engineers, data scientists, and More ❯

Senior Data Engineer – London - Hybrid Mars is a family-owned business with more than $35 billion in global sales. We produce some of the world’s best-loved brands: M&M’s®, SNICKERS®, TWIX®, MILKY WAY®, DOVE®, PEDIGREE®, ROYAL CANIN®, WHISKAS®, EXTRA®, ORBIT®, 5TM, SKITTLES®, BEN’S ORIGINAL®, and COCOAVIA®. Alongside our consumer brands, we proudly take … A capabilities and maintain them. Partnerships with other teams such as Supply, Manufacturing & R&D are also key to ensure process alignment and shared value creation. As a Data Engineer in the Commercial team, your key responsibilities are as follows: 1. Technical Proficiency: Collaborate in hands-on development using Python, PySpark, and other relevant technologies to create and … and Databricks. Proficiency in working with the cloud environment and various platforms, including Azure, SQL Server. NoSQL databases is good to have. Hands-on experience with data pipeline development, ETL processes, and big data technologies (e.g., Hadoop, Spark, Kafka). Experience with DataOps practices and tools, including CI/CD for data pipelines. More ❯

Senior Data Engineer – London - Hybrid Mars is a family-owned business with more than $35 billion in global sales. We produce some of the world’s best-loved brands: M&M’s®, SNICKERS®, TWIX®, MILKY WAY®, DOVE®, PEDIGREE®, ROYAL CANIN®, WHISKAS®, EXTRA®, ORBIT®, 5TM, SKITTLES®, BEN’S ORIGINAL®, and COCOAVIA®. Alongside our consumer brands, we proudly take … A capabilities and maintain them. Partnerships with other teams such as Supply, Manufacturing & R&D are also key to ensure process alignment and shared value creation. As a Data Engineer in the Commercial team, your key responsibilities are as follows: 1. Technical Proficiency: Collaborate in hands-on development using Python, PySpark, and other relevant technologies to create and … and Databricks. Proficiency in working with the cloud environment and various platforms, including Azure, SQL Server. NoSQL databases is good to have. Hands-on experience with data pipeline development, ETL processes, and big data technologies (e.g., Hadoop, Spark, Kafka). Experience with DataOps practices and tools, including CI/CD for data pipelines. More ❯

Data Engineer - Greater Manchester - Hybrid - £55k - £65k I am working with a financial services client, a forward-thinking organization using data to drive innovation and business performance. They’re expanding their team and are looking for a talented Data Engineer with experience in Azure and Databricks to join the team. Salary and Benefits … the Greater Manchester office Supportive team and collaborative culture Ongoing professional development opportunities 28 days holiday + bank holidays Pension and other core benefits The Role: As a Data Engineer, you'll play a key role in designing, building, and maintaining high-quality data pipelines and architectures that support our business intelligence and analytics goals. You … ll work closely with data scientists, analysts, and other engineers to deliver reliable and performant data solutions. Key Responsibilities: Develop and manage scalable data pipelines using Azure Data Factory, Data Lake, Synapse, and Databricks. Build and maintain Delta Lake architectures to support both batch and real-time processing. Collaborate with More ❯

in the digital lifecycle and helping them outperform their competition through speed and innovation. Apexon brings together distinct core competencies – in AI, analytics, app development, cloud, commerce, CX, data, DevOps, IoT, mobile, quality engineering and UX, and our deep expertise in BFSI, healthcare, and life sciences – to help businesses capitalize on the unlimited opportunities digital offers. Our reputation … Everstone Capital, Apexon now has a global presence of 15 offices (and 10 delivery centers) across four continents. About the Role: We are currently looking for a Senior Data Engineer to manage teams on client projects. Your Profile: Proven experience as a Data Engineer with a focus on Azure cloud services. Experience of managing small teams … Factory, Azure Synapse Analytics, Azure Data Lake, Azure Databricks and Power BI. Experience with creating low-level designs for data platform implementations. ETL pipeline development for the integration with data sources and data transformations including the creation of supplementary documentation. Proficiency in working with APIs and integrating them into More ❯

for new, innovative and sustainable products and solutions, both for today and for a new era of building. To support our progress, we are currently recruiting for a Data Engineer (Databricks & BI) to come and join our team at Ibstock Head Office, LE67 6HS. (Applicants must have Right to work in the UK - We are unable to provide … sponsorship for this role) Job Purpose: The Data Engineer/BI Developer will play a critical role in developing and refining our modern data lakehouse platform, with a primary focus on Databricks for data engineering, transformation, and analytics. The role will involve designing, developing, and maintaining scalable data solutions to support business … and on-premises JDE systems to provide near real-time insights for decision-making. Key Accountabilities: Ensure the existing design, development, and expansion of a near real-time data platform integrating AWS, Databricks and on-prem JDE systems. Develop ETL processes to integrate and transform data from multiple sources into a centralised data platform. More ❯

through the Medallion architecture in Databricks . Data Transformation : Create and manage data models and transformations using dbt , ensuring clear lineage, version control, and modularity. Pipeline Orchestration : Develop and manage workflow orchestration using Azure Data Factory , including setting up triggers, pipelines, and integration runtimes. System Maintenance : Monitor, maintain, and optimize existing data … including notebooks, cluster configuration, and Delta Lake. Proficiency in dbt for transformation logic and version-controlled data modeling. Deep knowledge of Azure Data Factory , including pipeline orchestration and integration with other Azure services. Experience with data integration (e.g. : APIs, JSON, XML, Web Services) essential Expertise in Python and SQL for data manipulation and pipeline development. Hands-on experience implementing and maintaining Medallion Architecture (Bronze/Silver/Gold). Familiarity with CI/CD, Git version control, and agile development methodologies. Strong understanding of data warehousing principles, data modeling, and performance optimization. Experience with cron jobs , job orchestration, and error monitoring tools. Good to More ❯

Job Title: Data Engineer Location: Edinburgh, Scotland (Hybrid 2 days on-site) Job Type: Full-Time, Permanent Salary: £35K About Us We’re a forward-thinking IT consultancy based in the heart of Edinburgh, delivering innovative data-driven solutions to clients across finance, healthcare, energy, and the public sector. Our mission is to empower businesses through … smart data engineering, cloud transformation, and cutting-edge analytics. The Role We’re looking for a talented and motivated Data Engineer to join our growing team. You’ll play a key role in designing, building, and maintaining robust data pipelines and architectures that support our clients’ strategic goals. Key Responsibilities Design and implement scalable … all projects. What We’re Looking For Proven experience as a Data Engineer or in a similar role. Strong knowledge of SQL, Python, and data pipeline tools (e.g., Apache Airflow, dbt). Experience with cloud services (AWS, Azure, or GCP). Familiarity with data warehousing solutions (e.g., Snowflake, Redshift, BigQuery). Excellent problem More ❯

To Work. This prestigious award is based entirely on what current employees say about their experience working at Circana. Job description: We are seeking a skilled and motivated Data Engineer to join a growing global team. In this role, you will be responsible for designing, building, and maintaining robust data pipelines and infrastructure on the Azure … cloud platform. You will leverage your expertise in PySpark, Apache Spark, and Apache Airflow to process and orchestrate large-scale data workloads, ensuring data quality, efficiency, and … scalability. If you have a passion for data engineering and a desire to make a significant impact, we encourage you to apply! Job Responsibilities ETL/ELT Pipeline Development: Design, develop, and optimize efficient and scalable ETL/ELT pipelines using Python, PySpark, and Apache Airflow. Implement batch and real-time data processing solutions using More ❯

constituent of the FTSE 100 index of the London Stock Exchange. The Position Our roster has an opening with your name on it! We are seeking a Staff Data Engineer to lead the technical design and implementation of our most critical data infrastructure and products. In this senior-level individual contributor role, you'll be responsible … for designing scalable systems, setting data architecture standards, and solving complex technical challenges that power analytics, data science, and business function use cases across the company. You will collaborate closely with engineers, product managers, and business stakeholders to architect data solutions that are performant, reliable, and built with a long-term, customer-centric mindset. … batch & streaming data pipelines and architectures that support enterprise-wide data needs Define standards for data modeling, data product design, and pipeline orchestration using modern tools and cloud-native technologies Collaborate with cross-functional stakeholders to translate business and analytical requirements into end-to-end data solutions Establish and More ❯

Role Overview: We are seeking a highly skilled and experienced Senior Azure Data Engineer to join our team. In this role, you will lead the design, development, and maintenance of scalable data solutions in Microsoft Azure. You will work closely with cross-functional teams to transform complex data into actionable insights, driving business decisions … and innovation. This is a remote working position with only occasional visits to the office, which is based in Burton-on-Trent, Staffordshire. Key Responsibilities: Data Architecture and Design: Design, develop, and implement end-to-end data pipelines in Azure. Architect data solutions using Azure Data Lake, Azure Synapse Analytics, and Azure … services such as Azure Data Factory, Azure Data Lake, Azure Databricks, Azure Synapse Analytics, and Azure SQL Database. Strong experience with ETL/ELT pipelines, data modelling, and data integration. Proficiency in SQL and programming languages like Python, Scala, or PowerShell. Knowledge of data governance, security, and compliance frameworks. Preferred More ❯

Senior Data Engineer – Azure Platform Overview Amtis is proud to be recruiting for an exciting opportunity to join a forward-thinking data team. This role is ideal for a technically skilled and proactive data engineer passionate about building scalable, secure, and high-performing data solutions in Azure. You’ll play a key … role in shaping the data strategy, enhancing platform capabilities, and supporting business intelligence initiatives. Key Responsibilities Design and develop Azure data pipelines using Data Factory, Databricks, and related services. Implement and optimize ETL processes for performance, reliability, and cost-efficiency. Build scalable data models and support analytics and reporting needs. Design and … maintain Azure-based data warehouses and data lakes. Ensure data governance, security, and compliance with GDPR and other regulations. Collaborate with cross-functional teams to gather requirements and deliver solutions. Manage Azure SQL databases, including performance tuning and monitoring. Monitor and maintain data infrastructure, resolving issues proactively. Document technical designs and share More ❯

Senior Data Engineer – Azure Platform Overview Amtis is proud to be recruiting for an exciting opportunity to join a forward-thinking data team. This role is ideal for a technically skilled and proactive data engineer passionate about building scalable, secure, and high-performing data solutions in Azure. You’ll play a key … role in shaping the data strategy, enhancing platform capabilities, and supporting business intelligence initiatives. Key Responsibilities Design and develop Azure data pipelines using Data Factory, Databricks, and related services. Implement and optimize ETL processes for performance, reliability, and cost-efficiency. Build scalable data models and support analytics and reporting needs. Design and … maintain Azure-based data warehouses and data lakes. Ensure data governance, security, and compliance with GDPR and other regulations. Collaborate with cross-functional teams to gather requirements and deliver solutions. Manage Azure SQL databases, including performance tuning and monitoring. Monitor and maintain data infrastructure, resolving issues proactively. Document technical designs and share More ❯

Job summary Join NHSBSA as an Apprentice Data Engineer and gain hands-on experience building and maintaining data pipelines while studying for a Level 5 qualification. You'll support data sourcing, transformation, and integration under expert guidance, ensuring data is accurate, secure, and accessible. Collaborate with analysts, engineers, and governance teams to … deliver data solutions. You'll develop coding, documentation, and troubleshooting skills, and apply data protection principles. This hybrid role includes remote work and regular office visits in Newcastle. Ideal for someone curious, detail-oriented, and eager to learn in a supportive, innovative environment. What do we offer? o Hybrid working - offering flexibility to work predominantly from … your loved oneso Access to a wide range of benefits and high street and online discounts Main duties of the job Support the design, build, and testing of data pipelines using multiple data sources. Assist in creating reusable data products and maintaining accurate documentation. Learn and apply data protection and governance principles. More ❯

An established organisation with a strong data-driven culture is seeking an experienced Data Engineer or Data specialist to play a key role in shaping the future of its business intelligence and analytics capabilities. As a skilled and versatile senior Data professional you will bridge the gap between data engineering … and business intelligence. This role is ideal for someone who is equally comfortable designing complex data infrastructure and building intuitive, insight-driven dashboards. You’ll play a key role in the full lifecycle of data - from ingestion and modelling through to reporting and decision support - helping shape the way our organisation uses data to … drive performance and strategy. What You’ll Do: Back-End Architecture & Modelling Design and implement scalable, high-performance data warehouse and data lake architectures. Develop conceptual, logical, and physical data models to support analytical requirements. Build and optimise data pipelines (ETL/ELT) using tools such as Azure Synapse, Snowflake, Redshift, or More ❯