76 to 100 of 2,576 Data Pipeline Jobs in the UK

team you'll be working with: We are seeking an experienced QE Automation Lead to operate as the Test Lead for automated data testing within a complex, multi‐vendor data transformation programme. This test role reports to the QE Lead (Test Director level) and is accountable … logic and data queries for data transformationsTest framework and supporting toolingLead testing of AWS‐based Lakehouse and data pipeline solutions, including: AWS GlueApache IcebergAI‐enabled ETL acceleratorsNeo4jPython/PySpark, SQL and YAML‐driven configurationsQuality, Defect & Risk ManagementEnsure quality issues, risks issues and data ...

institutions to measure and manage market risk, counterparty credit risk, regulatory risk capital, and derivative valuation adjustments. Задачи Work with product, financial engineering, data, technology, support, professional services, and client-facing teams to analyze requirements and design robust workflows for market data ingestion, enrichment, validation, transformation … integration, and delivery into Risk Analytics products and services Translate business, analytical, and operational needs into clear specifications, data mappings, process flows, user stories, acceptance criteria, test scenarios, release notes, and operational documentation Support the design and continuous improvement of data pipelines, APIs, integration services, controls ...

Bank House, Birmingham, LondonApply for this job* Permanent* Experienced Professionals* Architecture* ID 467117-en\_GB## **Who You'll Be Working With**As a Data Solution Architect at Capgemini, you will design and lead the delivery of modern data platforms that enable analytics, AI, GenAI, and data … Fabric experiences to operationalise models and drive business insights.* Design, monitor and optimise data pipelines and platform performance: Use Fabric capacity management, pipeline monitoring, logging, and operational best practices to improve reliability, throughput, cost, and scalability.* Collaborate with cross-functional teams: Work closely with business analysts, data ...

mappings, business rules, and transformation logic into automated test scenariosDevelop reusable and maintainable test scripts using approved automation frameworksTest data feed pipeline migrations repointing from an existing Data Warehouse to a new AWS Data Lakehouse architecture, underpinned with AI ETL accelerators, Python … closely with data engineers and analysts to diagnose issues and verify fixesSupport root-cause analysis of data quality failures and pipeline issuesCI/CD & Delivery SupportExecute automated tests within CI/CD pipelines and scheduled testing runsEnsure test assets are version controlled and aligned with ...

sends, logistics, offers, and reporting, but also the analytical core that makes it work, building the response model and sourcing the data pipeline and user-data enrichment that let us target the right customer, with the right offer, at the right moment. … creative, modelling, automation)Desirable: Experience in financial services, regulated or other consumer categories where offers and eligibility matterExperience standing up a data pipeline or enrichment stack (first-party + third-party/bureau data)Experience building a channel or function from scratchFamiliarity with credit/ ...

sends, logistics, offers, and reporting, but also the analytical core that makes it work, building the response model and sourcing the data pipeline and user-data enrichment that let us target the right customer, with the right offer, at the right moment. … creative, modelling, automation)Desirable: Experience in financial services, regulated or other consumer categories where offers and eligibility matterExperience standing up a data pipeline or enrichment stack (first-party + third-party/bureau data)Experience building a channel or function from scratchFamiliarity with credit/ ...

place where you can grow, belong and thrive. Your day at NTT DATA Primary Responsibilities Client Engagement & Delivery Data Pipeline Development (Batch and Streaming) Databricks & Lakehouse Architectures Data Modelling & Optimisation (Delta Lake, Medallion architecture) Collaboration & Best Practices Quality, Governance & Security Business Relationships Solution … Architect, and Analytics leadership To thrive in this role, you need to have: Key Competencies Proven experience in data engineering and pipeline development on Databricks and cloud-native platforms. Strong consulting values with ability to collaborate effectively in client-facing environments. Hands-on expertise across ...

place where you can grow, belong and thrive. Your day at NTT DATA Primary Responsibilities Client Engagement & Delivery Data Pipeline Development (Batch and Streaming) Databricks & Lakehouse Architectures Data Modelling & Optimisation (Delta Lake, Medallion architecture) Collaboration & Best Practices Quality, Governance & Security Business Relationships Solution … Architect, and Analytics leadership To thrive in this role, you need to have: Key Competencies Proven experience in data engineering and pipeline development on Databricks and cloud-native platforms. Strong consulting values with ability to collaborate effectively in client-facing environments. Hands-on expertise across ...

with engineering best practices and the ability to translate customer expectations into applied technical functionality are key to success in this role. Key responsibilitiesData pipeline development Build and maintain scalable data pipelines using Apache Spark and PySpark, processing and transforming large datasets across distributed cloud infrastructure. Workflow … monitoring, and execution of data processing workflows. Root cause analysis Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues - including reviewing EMR output logs and CloudWatch metrics. Data modelling Apply understanding of dimensional data ...

description Connect to your Industry Do you want to help deliver some of the UK's largest and most ambitious Data & AI transformation programmes? Technology & Transformation helps organisations unlock the full value of their data through modern platforms, cloud technologies, and AI. Within Deloitte's Engineering … team, you will: Design, build, test and deploy scalable batch and streaming data pipelines, AI-ready data products, feature pipeline and analytics solutions on the Databricks Platform using Delta Lake, Spark SQL/PySpark, Unity Catalog and Databricks SQL. Deliver solutions across the full data ...

sends, logistics, offers, and reporting, but also the analytical core that makes it work, building the response model and sourcing the data pipeline and user-data enrichment that let us target the right customer, with the right offer, at the right moment. … modelling, automation)**Desirable:*** Experience in financial services, regulated or other consumer categories where offers and eligibility matter* Experience standing up a data pipeline or enrichment stack (first-party + third-party/bureau data)* Experience building a channel or function from scratch* Familiarity with credit ...

Senior Data Engineer Make a difference and advance your career by helping deliver some of the UK’s most important #tech4good projects, making the world a smarter, safer, greener, and healthier place. You’ll join a talented team of dynamic and driven professional problem solvers; creative thinkers … digital transformation consultancy. We deliver multi-Queen’s Award for innovation winning platforms and services that support large-scale digital transformation. Our digital, data and technology solutions are used by globally recognised public and private sector brands operating in a variety of sectors including Civil Defence, Healthcare, Sustainable ...

Senior Data Engineer – Customer Data Location: Waterloo, London – Hybrid Contract: 6 months Rate: £650–£700 per day, Inside IR35 Working Pattern: Full-time About the Role We are seeking an experienced Senior Data Engineer to join the Customer Data team within … data and AI function of a leading UK retail organisation. The team is responsible for building and operating the data foundations that support a wide range of customer-focused use cases, including Customer 360, analytics, AI/ML, marketing and data-driven commercial initiatives ...

Data Engineer Department: [SBSS] Enterprise Data Management Employment Type: Permanent - Full Time Location: Newcastle Compensation: £53,000 - £67,000/year Description At Somerset Bridge Group, we're transforming our data platform to create a modern, cloud-based data ecosystem that powers … smarter decision-making across the business. As part of this journey, we're consolidating our data capabilities onto a single Azure Databricks platform, leveraging technologies including Azure Data Factory, Delta Lake, Unity Catalog and Power BI to unlock new insights and drive innovation. Joining our Enterprise ...

Talan Data x AI is a leading Data Management and Analytics consultancy, working closely with leading software vendors and top industry experts across a range of sectors, unlocking value and insight from their data. At Talan Data x AI, innovation is at the heart … everything we do, and we have been recertified as a 2025 Great Place to Work®. This achievement not only highlights Talan Data x AI’s positive organisational culture but also strengthens its reputation as an employer of choice within the industry. We invest heavily in the training ...

More about the role Our Data & Analytics team creates value for Morrisons by making sure we have the right data in the right place, transformed into clean, accessible structures ready for reporting, data product creation and enabling advanced decision-making. Utilising our Google Cloud … data platform, the Analytics Engineer transforms raw data from across our operations from field to fork into optimised, reliable, and well-documented data models and data products. You will bridge the gap between technical data pipelines and business insight, enabling ...

Data Engineer £60,000 - £70,000 DOE | East Kent | Generous Hybrid Working Are you an experienced Data Engineer looking for an opportunity to help shape and improve the way a growing international organisation uses data? We're working with a well-established, technology … organisation that is investing in its data and analytics capability. They are looking for a Data Engineer to join their Data Analytics team and play a key role in building reliable, scalable and reusable data solutions. This is an excellent opportunity ...

seeking a Data Engineering Specialist to join the Architecture & Modernization team. You will be instrumental in building … maintaining the data infrastructure for our Data AI platforms. This role will involve hands-on development, data pipeline creation, and close collaboration with stakeholders across the organization. This role requires a self-starter with strong execution skills and the ability to work independently. ...

Data Pipeline Manager Vacancy Type: Full Time, Permanent Location: Stratford-upon-Avon (Hybrid) Salary: Competitive + Up to 10% Bonus and Benefits Why choose Valpak? Valpak has created a friendly and supportive working environment that enables our team to perform to the best of their abilities, with … improve accuracy and utilise the data to improve knowledge within the sector. What will I be doing? As the Data Pipeline Manager, you will lead a team responsible for building and maintaining scalable, reliable data pipelines that support enterprise reporting and analytics. ...

world’s largest asset class at over $145 trillion. Debt markets are vast, global, and mission-critical, yet still run on fragmented data, PDFs, and manual workflows. 9fin replaces this broken infrastructure with a single platform that centralises proprietary credit data, deep analysis, and high-value … markets worldwide. The Opportunity 9fin has built one of the most comprehensive datasets in credit and debt markets: news, financials, cap tables, instrument data, legal documents, and ESG analysis. Today, most clients access that data through 9fin.com. But a large and growing segment of the market ...

OVERVIEW:This role exists to provide technical leadership within the data engineering team, setting the standard for engineering excellence while remaining deeply hands-on. The Lead Data Engineer drives the design and delivery of scalable, resilient data platforms and pipelines that underpin Gymshark … Data Product, and wider Tech teams to ensure joined up platform delivery. Maintain accurate and comprehensive technical documentation: architecture decision records, runbooks, pipeline specs, and data dictionaries. Uphold data governance, security, and compliance standards across all data engineering activities. WHAT ...

shaping the future of gaming. Are you ready to LEVEL-UP THE GAME? Focusing on our Sportsbook product, we’re looking for a Data Engineer to help build and evolve our next-generation data platform, powering trusted, scalable, and near real-time data assets … Spark) and workflow orchestration (e.g. Airflow or similar). Familiarity with cloud-based data platforms, ideally AWS. Understanding of data pipeline design and ETL/ELT patterns. Exposure to streaming technologies (Kafka, Flink, or similar) is beneficial. Awareness of data quality, testing ...

NegotiableReading, England Contract Principal Data Engineer - Secure Digital Platform (SDP) Data and Integration Location: Reading (hybrid working) Duration: Initially Until Feb '27 - but very likely to extend (new programme)(Inside IR35)Active SC Clearance Required Role details: We are looking for SC cleared Data Engineers to support our defence client on a contract basis. This role is focused on the Secure Digital Platform (SDP) Data and Integration, and will be based in Reading with hybrid working arrangements. The initial contract will run until February 2027, with a strong likelihood ...

work depends on, including taking our golden dataset of completed matters from a manual exercise to a versioned, access-controlled, anonymised and repeatable pipeline, along with the evaluation data that tells us whether our AI is getting better. Replace a legacy management information warehouse built on stored … tuning, data modelling and stored procedures, with the judgement to understand the impact of querying production systems. Python for data pipeline and automation work. Experience owning production data pipelines end-to-end: built them, monitored them, fixed them when they broke. Experience working ...

small team with a large remit, and we expect AI-assisted development to be embedded in how we work Apply AI tools to accelerate pipeline development, automated testing, anomaly detection and data quality monitoring Stay current with AI developments in the data engineering space … this is non‐negotiable; you must have built and operated production data pipelines on this stack Strong Python skills – used daily for pipeline development, data transformation and automation Experience across at least two of the three core data domains: Customer, Commercial and Operations ...