26 to 50 of 877 Data Pipeline Jobs in the UK

Join our client in embarking on an ambitious data transformation journey using Databricks, guided by best practice data governance and architectural principles. To support this, we are recruiting for talented data engineers. As a major UK energy provider, our client is committed to 100% renewable energy and sustainability, focusing on delivering exceptional customer experiences. … day a week being based in their Nottingham office, this is negotiable. It is a full-time role, 37 hours per week. Accountabilities: * Develop and maintain scalable, efficient data pipelines within Databricks, continuously evolving them as requirements and technologies change. * Build and manage an enterprise data model within Databricks. * Integrate new data sources into … such as data warehousing. * Able to write repeatable, production-quality code for data pipelines, utilizing templating and parameterization where needed. * Can make data pipeline design recommendations based on business requirements. * Experience with data migration is a plus. * Open to new ways of working and new technologies. * Self-motivated with the ability More ❯

Gett, Kaizen Gaming, and TransferGo. Our products are recognised by industry leaders like Gartner's Magic Quadrant, Forrester Wave and Frost Radar. Our tech stack: Superset and similar data visualisation tools. ETL tools: Airflow, DBT, Airbyte, Flink, etc. Data warehousing and storage solutions: ClickHouse, Trino, S3. AWS Cloud, Kubernetes, Helm. Relevant programming languages for data engineering tasks: SQL, Python, Java, etc. What you will be doing: Designing and developing scalable and efficient data pipelines, ETL processes, and data integration solutions to support data ingestion, processing, and storage needs. Ensuring data quality and reliability by implementing data validation, data cleansing, and data … quality monitoring processes. Optimising database performance by tuning queries, implementing indexing strategies, and monitoring and analysing system performance metrics. Collaborating with cross-functional teams to gather requirements, understand data needs, and develop data solutions that align with business objectives. Staying up-to-date with emerging technologies and industry trends in data engineering, and identifying More ❯

Purpose of the Job: Design, build, and maintain robust data systems and pipelines that support data storage, processing, and analysis on the Cloud. Work with large datasets, ensuring data quality, scalability, and performance, while collaborating. closely with data scientists, analysts, and other engineering teams to understand their data needs and … provide them with high-quality, accessible data. They are responsible for ensuring that the underlying data infrastructure supports the organizations broader data and business goals, enabling more effective data-driven decision-making. Key Accountabilities: • Design and implement scalable, efficient, and secure data architectures, ensuring optimal data flow across systems in … and process across the team along with BI lead. • Influence the evolution of business and system requirements and contribute to the design of technical solutions to feed a delivery pipeline that increasingly employs Agile methods such as SCRUM and Kanban • You will be required to develop unit tested code and then support test cycles including post implementation validation. You More ❯

Senior Data Engineer - OnTheMarket Technology - London Job Description COSTAR GROUP - SENIOR DATA ENGINEER - ONTHEMARKET TECHNOLOGY - LONDON OVERVIEW CoStar Group (NASDAQ: CSGP) is a leading global provider of commercial and residential real estate information, analytics, and online marketplaces. Included in the S&P 500 Index and the NASDAQ 100, CoStar Group is on a mission to digitize … content, imagery, design and functionality. The OnTheMarket product will support home buyers, sellers, and agents better than ever! Learn more about OnTheMarket. ROLE DESCRIPTION The role of Senior Data Engineer will be a key role within the OTM Data team, reporting to the Head of Data and working closely with the CTO. The Data team is responsible for data acquisition, data loading, data quality, data processing, data delivery, and data support for OnTheMarket. This role will provide focus on delivering value to our customers, by leveraging data to improve the OTM users and agents experience. This role would More ❯

other categories like cakes, candles and cocktails. This is all part of our ambition to build Europe's direct to consumer gifting destination. ️ The Business Intelligence & Analytics & Wild Data is at the heart of what we do at Bloom & Wild Group. Our BI & Analytics team empowers growth by transforming data into actionable insights and we're … passionate about understanding customer behavior and identifying opportunities to delight them even more. We leverage SQL, data visualisation, and rigorous experimentation and analysis to uncover trends, predict user needs, and test hypotheses. We believe data drives positive change and empowers informed decisions and we value collaboration, continuous learning, and a data-driven mindset. Please … making them more cost effective & run faster. Implement DBT best practices for modularity, reusability, and incremental builds to significantly reduce Snowflake compute and storage costs. Monitor data pipeline performance and identify areas for optimisation to ensure efficient data processing. Create New Scalable Data Models: Develop new, robust, and well-documented data More ❯

with a flexible, student-powered workforce. The platform enables partnersfrom local pubs to multinational chainsto optimise staffing in real-time, enhancing efficiency and profitability. The Role As a Data Engineer, you'll play a pivotal role in shaping the data infrastructure. Your responsibilities will include: Building and Maintaining Data Pipelines: Design and implement scalable … data scientists, software engineers, and internal stakeholders to understand data needs and deliver solutions that drive data-driven decisions. Monitoring and Troubleshooting: Oversee pipeline performance, address issues promptly, and maintain comprehensive data documentation. What Youll Bring Technical Expertise: Proficiency in Python and SQL; experience with data processing frameworks such … architecture, data modelling, and scalable data solutions. Backend Development: Willingness to develop proficiency in backend technologies (e.g., Python with Django) to support data pipeline integrations. Cloud Platforms: Familiarity with AWS or Azure, including services like Apache Airflow, Terraform, or SageMaker. Data Quality Management: Experience with data versioning and quality More ❯

Data Engineer Sr - Informatica ETL Expert page is loaded Data Engineer Sr - Informatica ETL Expert Apply locations Two PNC Plaza (PA374) Birmingham - Brock (AL112) Dallas Innovation Center - Luna Rd (TX270) Strongsville Technology Center (OH537) time type Full time posted on Posted 3 Days Ago job requisition id R185351 Position Overview Job Description Summary At PNC, our … day to foster an inclusive workplace culture where all of our employees feel respected, valued and have an opportunity to contribute to the company's success. As a Data Engineer within PNC's Technology organization, you will be based in Pittsburgh, PA; Cleveland, OH; Birmingham, AL or Dallas, TX. The position is primarily based in a PNC location. … manager's discretion. PNC will not provide sponsorship for employment visas or participate in STEM OPT for this position Key responsibilities - - Design, Develop and maintain scalable and efficient data pipelines and ETL processes - Collaborate with analysts and stakeholders to gather requirements and implement data solutions that meet business needs - Proposes & designs data solutions to More ❯

Senior data engineers at Thoughtworks are engineers who build, maintain and test the software architecture and infrastructure for managing data applications. They are involved in developing core capabilities which include technical and functional data platforms. They are the anchor for functional streams of work and are accountable for timely delivery. They work on the … latest big data tools, frameworks and offerings (data mesh, etc.), while also being involved in enabling credible and collaborative problem solving to execute on a strategy. Job responsibilities You will develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions You will develop … intricate data processing pipelines, addressing clients' most challenging problems You will collaborate with data scientists to design scalable implementations of their models You will write clean, iterative code using TDD and leverage various continuous delivery practices to deploy, support and operate data pipelines You will use different distributed storage and computing technologies from the More ❯

Senior data engineers at Thoughtworks are engineers who build, maintain and test the software architecture and infrastructure for managing data applications. They are involved in developing core capabilities which include technical and functional data platforms. They are the anchor for functional streams of work and are accountable for timely delivery. They work on the … latest big data tools, frameworks and offerings (data mesh, etc.), while also being involved in enabling credible and collaborative problem solving to execute on a strategy. Job responsibilities You will develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions You will develop … intricate data processing pipelines, addressing clients' most challenging problems You will collaborate with data scientists to design scalable implementations of their models You will write clean, iterative code using TDD and leverage various continuous delivery practices to deploy, support and operate data pipelines You will use different distributed storage and computing technologies from the More ❯

Description Continuing Defleet's data led journey and recognising the importance the Data & Analytics team performs in this regard, we are looking for a skilled Lead Data Engineer who is passionate about exploiting data and searching for improvements in our existing processes and technology; can work closely with business teams; drives solutions … a focused mentality on business value and can lead, influence and drive collaboration across teams and stakeholders. It will focus on expanding the existing mission critical ecosystem of Data and BI systems for analytics and decision making, using Primarily Data Warehouse/Mart, Data Lake, Self Service BI Tools and Data Science … projects to tight deadlines Ability to work unsupervised and to manage a varied workload efficiently Minimum criteria You'll need all of these. Solid understanding of data pipeline/ELT development, including performance tuning Very good SQL skills Past experience and solid understanding of end to end Data Warehouse, Data Lake and BI More ❯

Description Continuing Defleet's data led journey and recognising the importance the Data & Analytics team performs in this regard, we are looking for a skilled Lead Data Engineer who is passionate about exploiting data and searching for improvements in our existing processes and technology; can work closely with business teams; drives solutions … a focused mentality on business value and can lead, influence and drive collaboration across teams and stakeholders. It will focus on expanding the existing mission critical ecosystem of Data and BI systems for analytics and decision making, using Primarily Data Warehouse/Mart, Data Lake, Self Service BI Tools and Data Science … projects to tight deadlines Ability to work unsupervised and to manage a varied workload efficiently Minimum criteria You'll need all of these. Solid understanding of data pipeline/ELT development, including performance tuning Very good SQL skills Past experience and solid understanding of end to end Data Warehouse, Data Lake and BI More ❯

Data Engineer Lead - NatGas and Power page is loaded Data Engineer Lead - NatGas and Power Apply locations London posted on Posted 12 Days Ago job requisition id JR102268 Job Title: Data Engineer Lead - NatGas and Power Contract Type: Time Type: Job Description: Reporting to the Head of Data, the Data Engineering Lead will oversee the design, development, and maintenance of high-performance data systems that support trading, risk, and analytics functions across the Natural Gas and Power business. This role bridges technical execution and business impact, ensuring that data pipelines, platforms, and products are robust, scalable, and aligned with enterprise data strategy. … The ideal candidate will lead a team of data engineers, collaborate with trading desks and analysts, and ensure that data infrastructure supports pre-trade modeling, live execution, and post-trade analytics with precision and reliability. Main Responsibilities 1. Data Platform Ownership Lead the development and maintenance of scalable data pipelines and databases More ❯

Senior Data Engineer Location: London, UK (3 days in the office) SC Cleared: Required Essential Skills & Experience: 10+ years of experience in data engineering, with at least 3+ years of hands-on experience with Azure Databricks. Strong proficiency in Python and Spark (PySpark) or Scala. Deep understanding of data warehousing principles, data modelling techniques, and data integration patterns. Extensive experience with Azure data services, including Azure Data Factory … Azure Blob Storage, and Azure SQL Database. Experience working with large datasets and complex data pipelines. Experience with data architecture design and data pipeline optimization. Proven expertise with Databricks, including hands-on implementation experience and certifications. Experience with SQL and NoSQL databases. Experience with data quality and data governance More ❯

Join Barclays as a Senior Cloud Data Platform Engineer , where you'll be responsible for building and maintaining systems that collect, store, process, and evaluate data-such as data pipelines, data warehouses, and data lakes-to ensure all data is accurate, accessible, and secure. To be successful in … this role, you should have experience with: Cloud platforms (AWS/Azure) Data Engineering (Airflow/DBT, Spark) DevSecOps practices Additional highly valued skills include: Terraform Python/Java AWS/Azure Data Engineering Snowflake/Databricks You may be assessed on key skills relevant for success in this role, such as risk and controls, change … as well as job-specific technical skills. This role is based in Glasgow. Purpose of the role To build and maintain systems that collect, store, process, and analyze data, such as data pipelines, data warehouses, and data lakes, ensuring data accuracy, accessibility, and security. Accountabilities Build and maintain data More ❯

Overview We're seeking a Senior Data Scientist to lead the development of advanced analytics and AI/ML solutions that unlock real value across our business. This is a contract role for 6 months . In this contract role, you'll work with proprietary and B2B research datasets to design, deliver, and scale data-driven … Research, and Technology, you'll help turn strategic ideas into working MVPs-ensuring high standards of methodology, quality, and business relevance throughout. You'll also help shape the data science environment by working alongside our tech teams to support a robust and flexible infrastructure, including sandbox environments for onboarding and evaluating new data sources. This is … refined and productionized in the future. Work with Tech colleagues to improve the Data Science workspace, including providing requirements for Data Lake, Data Pipeline, and Data Engineering teams. Expand on the tools and techniques already developed. Help us understand our customers (both internal and external) better so we can provide the More ❯

outlook are not only respected but expected. We want all our employees to feel that they can be themselves at work and that they are valued. Digital and Data The UKHSA Digital and Data Directorate provides strategic leadership and oversight for developing and managing a common and bespoke suite of performant, modern, secure, data and analytical tools, services, platforms and applications, ensuring access to high quality data and analytics services, internally and externally. There is a Market pay supplement available for this position of £10,000 up until the end of June 2026. Main duties of the job You will work in the Data Engineering and Enablement Division … that reports into the Digital and Data Directorate. Its an exciting time to join as we seek to create leading Health Protection data capabilities that enable data scientists and data analysts to develop insights that inform public health decisions with resilience platforms and curated data assets. The core activities of More ❯

outlook are not only respected but expected. We want all our employees to feel that they can be themselves at work and that they are valued. Digital and Data The UKHSA Digital and Data Directorate provides strategic leadership and oversight for developing and managing a common and bespoke suite of performant, modern, secure, data and analytical tools, services, platforms and applications, ensuring access to high quality data and analytics services, internally and externally. There is a Market pay supplement available for this position of £10,000 up until the end of June 2026. Main duties of the job You will work in the Data Engineering and Enablement Division … that reports into the Digital and Data Directorate. It's an exciting time to join as we seek to create leading Health Protection data capabilities that enable data scientists and data analysts to develop insights that inform public health decisions with resilience platforms and curated data assets. The core activities More ❯

Who We Are Graphite is a specialist team within Singulier focused exclusively on data transformation for PE-backed companies . We believe mid-sized and large companies are still underserved when it comes to data . For years, only big corporates could afford the platforms and teams needed to truly leverage their data. But over the … are short-cycle, high-impact, and always business-driven . We don't do "data for data's sake", every data product or pipeline we build has a direct line to a business decision. That's what makes us trusted by some of the best private equity funds in Europe. What You'll … work with you to remove blockers and deliver impact. Drive adoption : make sure the products you build are actually used, whether it's a dashboard, a data pipeline, or a reporting layer, by training and supporting business teams. Test and iterate : work with a hypothesis-driven approach, refining both the business framing and the data More ❯

with successful projects in key government departments like the Department for Work and Pensions, Ministry of Justice, HM Passport Office, and more. Overview We're seeking a seasoned Data Lead to own and deliver the data strategy for Govt client that uses services used by over 18m citizens encompassing three newly rebuilt benefits systems. You will … define where our data capabilities are today, set the future vision, and architect the roadmap to get us there-then design the data team to execute it. Key Responsibilities Lead the end-to-end Data Strategy for Bereavement & Care services. Map the "as-is" and "to-be" state of our data platforms … strategies in complex organisations. Technical Breadth : Cloud-based data platforms AWS must (Azure, AWS or equivalent) Modern ETL/ELT tools and data pipeline frameworks Data modelling, warehousing and transformation best practices Data science/ML lifecycle from prototype to production Government/Public Sector : Experience to GDS-equivalent More ❯

thefoundation for delivering our full suite of capabilities. Job Description Employment Type: Full-Time Location: UK Job Overview We are looking for a skilled Database Designer/Senior Data Engineer to join our data engineering team. This role involves the development and maintenance of data solutions in the cloud environment. The ideal candidate will … have a strong background in data processing and a solid understanding of cloud data services. Requirements Key Responsibilities Design and implement modern data platforms in cloud such as AWS or Azure. Develop data pipelines and data streaming processes. Ensure data solutions are scalable, reliable, and efficient. Collaborate with … data analysts and other stakeholders to meet data requirements. Optimize data storage and processing for performance and cost. Maintain data security and compliance standards. Minimum Qualifications Bachelor's degree in Computer Science , Information Technology, Engineering, or a related field. Minimum 3 years of experience as a Database Designer & Senior Data More ❯

Embark on a transformative journey with Barclays as a Data Security Lead to play a pivotal role in building and maintaining the secure systems that collect, store, process, and analyse data across our organisation. This includes safeguarding data pipelines, data warehouses, and data lakes-ensuring all information is accurate, accessible … and, above all, secure. As a Data Security Lead, you will collaborate with cross-functional teams to embed security by design into our data infrastructure, enabling advanced analytics and data science use cases-including AI and Machine Learning-without compromising on compliance or control. To be successful as a Data Security Lead … you should have experience with: Proven architecture and control design experience in a data security or cybersecurity context. Strong background as a Cloud technologist with hands-on experience in data science platforms and AI/ML use cases. Demonstrated expertise in implementing security by design principles and conducting threat modelling and translating this into designs mitigations More ❯

Azure Data Engineer Up to £74,000 + Benefits International Law Firm – London (Hybrid) I’m working with a highly respected international law firm that’s looking for a skilled and proactive Azure Data Engineer to join their forward-thinking Data team. This role is a great fit for someone who thrives on solving … complex data challenges and wants to directly influence how a global legal business uses its data to make smarter decisions. As part of the Strategy, Transformation & Project Delivery function, you’ll be instrumental in designing, developing, and optimising the firm’s Azure-based data infrastructure, from building scalable data pipelines to ensuring … data integrity, security, and compliance. The Role Data Infrastructure Development – Build and maintain scalable Azure data pipelines (following Senior Data Architect designs) using tools such as Azure Data Factory and Tray.io. Implement and manage data lakes and warehouses (Azure Synapse, Azure Data Fabric, etc.). Integration More ❯

progression, sustainability, or volunteering, you'll find like-minded people driving change at Howden. Role: Quality Assurance Engineer Objective: Ensure the quality, reliability, and governance of the Group Data Platform , covering data engineering, platform engineering, and data governance processes . Summary of the Role Howden Group Services is expanding its Group Data Platform and seeks a Quality Assurance Engineer to test, validate, and improve data pipelines, infrastructure, and governance processes. The ideal candidate will have experience in data testing, automation frameworks, and cloud-based data platforms , ensuring high-quality standards for data pipelines, storage, and platform components . This role requires a … processes . Develop and execute test cases for data ingestion, transformation, and storage . Implement automated testing frameworks to validate data integrity, schema consistency, and pipeline performance . Assist in testing platform engineering components , including CI/CD pipelines, infrastructure automation, and cloud-based services . Validate data governance policies , ensuring adherence to More ❯

health. EIT fosters a culture of collaboration, innovation, and resilience, valuing diverse expertise to drive sustainable solutions to humanity's enduring challenges. We are looking for a Senior Data Engineer to join the EIT Pathogen Program and play a key role in designing, implementing, and supporting the cloud-based data platform that underpins our mission. This … role offers the opportunity to shape the future of pathogen monitoring and diagnostics by building scalable, secure, and high-performance data pipelines. You'll work closely with cross-functional teams, including data architects, platform engineers, and product teams, to deliver the data infrastructure that powers advanced analytics and AI-driven solutions for global health … impact. Key Responsibilities: Ensure data in the platform is acquired, processed, curated, and made accessible to scientists, digital analytics products, bioinformatics, and AI at a high standard of quality and availability Ensure data access adheres to FAIR principles (Findable, Accessible, Interoperable, and Re-usable) Ensure data is secured and compliant with regulatory, legal, and More ❯

About the Role We are looking for a highly skilled Data Engineer … to support data integration, transformation, and orchestration initiatives for one of our leading telecom clients based in London. This role requires deep expertise in data pipeline development and data modeling, using tools such as DBT, Airflow, Fivetran, and Erwin. We need with minimum 8+ years of experience Key Responsibilities: Design, build, and maintain … scalable and robust data pipelines using Airflow, Fivetran, and DBT. Develop data models and document metadata using Erwin Data Modeler. Collaborate with business stakeholders and data analysts to understand data requirements and translate them into efficient engineering solutions. Optimize data flows and manage orchestration and scheduling using Apache More ❯