51 to 75 of 1,690 Data Pipeline Jobs in the UK

with engineering best practices and the ability to translate customer expectations into applied technical functionality are key to success in this role. Key responsibilitiesData pipeline development Build and maintain scalable data pipelines using Apache Spark and PySpark, processing and transforming large datasets across distributed cloud infrastructure. Workflow … monitoring, and execution of data processing workflows. Root cause analysis Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues - including reviewing EMR output logs and CloudWatch metrics. Data modelling Apply understanding of dimensional data ...

sends, logistics, offers, and reporting, but also the analytical core that makes it work, building the response model and sourcing the data pipeline and user-data enrichment that let us target the right customer, with the right offer, at the right moment. … modelling, automation)**Desirable:*** Experience in financial services, regulated or other consumer categories where offers and eligibility matter* Experience standing up a data pipeline or enrichment stack (first-party + third-party/bureau data)* Experience building a channel or function from scratch* Familiarity with credit ...

Senior Data Engineer – Customer Data Location: Waterloo, London – Hybrid Contract: 6 months Rate: £650–£700 per day, Inside IR35 Working Pattern: Full-time About the Role We are seeking an experienced Senior Data Engineer to join the Customer Data team within … data and AI function of a leading UK retail organisation. The team is responsible for building and operating the data foundations that support a wide range of customer-focused use cases, including Customer 360, analytics, AI/ML, marketing and data-driven commercial initiatives ...

Data Engineer £60,000 - £70,000 DOE | East Kent | Generous Hybrid Working Are you an experienced Data Engineer looking for an opportunity to help shape and improve the way a growing international organisation uses data? We're working with a well-established, technology … organisation that is investing in its data and analytics capability. They are looking for a Data Engineer to join their Data Analytics team and play a key role in building reliable, scalable and reusable data solutions. This is an excellent opportunity ...

mappings, business rules, and transformation logic into automated test scenariosDevelop reusable and maintainable test scripts using approved automation frameworksTest data feed pipeline migrations repointing from an existing Data Warehouse to a new AWS Data Lakehouse architecture, underpinned with AI ETL accelerators, Python … closely with data engineers and analysts to diagnose issues and verify fixesSupport root-cause analysis of data quality failures and pipeline issues CI/CD & Delivery Support Execute automated tests within CI/CD pipelines and scheduled testing runsEnsure test assets are version controlled ...

seeking a Data Engineering Specialist to join the Architecture & Modernization team. You will be instrumental in building … maintaining the data infrastructure for our Data AI platforms. This role will involve hands-on development, data pipeline creation, and close collaboration with stakeholders across the organization. This role requires a self-starter with strong execution skills and the ability to work independently. ...

Primary Responsibilities Client Engagement & Delivery Data Pipeline Development (Batch and Streaming) Databricks & Lakehouse Architectures Data Modelling & Optimisation (Delta Lake, Medallion architecture) Collaboration & Best Practices Quality, Governance & Security Business Relationships Solution Architects Data Engineers, Developers, ML Engineers, and Analysts Client stakeholders up to Head … Data Engineering, Chief Data Architect, and Analytics leadership Key Competencies Proven experience in data engineering and pipeline development on Databricks and cloud-native platforms. Strong consulting values with ability to collaborate effectively in client-facing environments. Hands‐on expertise across the data ...

Primary Responsibilities Client Engagement & Delivery Data Pipeline Development (Batch and Streaming) Databricks & Lakehouse Architectures Data Modelling & Optimisation (Delta Lake, Medallion architecture) Collaboration & Best Practices Quality, Governance & Security Business Relationships Solution Architects Data Engineers, Developers, ML Engineers, and Analysts Client stakeholders up to Head … Data Engineering, Chief Data Architect, and Analytics leadership Key Competencies Proven experience in data engineering and pipeline development on Databricks and cloud-native platforms. Strong consulting values with ability to collaborate effectively in client-facing environments. Hands‐on expertise across the data ...

world’s largest asset class at over $145 trillion. Debt markets are vast, global, and mission-critical, yet still run on fragmented data, PDFs, and manual workflows. 9fin replaces this broken infrastructure with a single platform that centralises proprietary credit data, deep analysis, and high-value … markets worldwide. The Opportunity 9fin has built one of the most comprehensive datasets in credit and debt markets: news, financials, cap tables, instrument data, legal documents, and ESG analysis. Today, most clients access that data through 9fin.com. But a large and growing segment of the market ...

shaping the future of gaming. Are you ready to LEVEL-UP THE GAME? Focusing on our Sportsbook product, we’re looking for a Data Engineer to help build and evolve our next-generation data platform, powering trusted, scalable, and near real-time data assets … Spark) and workflow orchestration (e.g. Airflow or similar). Familiarity with cloud-based data platforms, ideally AWS. Understanding of data pipeline design and ETL/ELT patterns. Exposure to streaming technologies (Kafka, Flink, or similar) is beneficial. Awareness of data quality, testing ...

Senior Data Engineer – Market Data We are looking for a Senior Data Engineer to build and operate a high-performance market data platform supporting quantitative research and production trading systems. You'll design scalable data pipelines, process billions of market … engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience ...

small team with a large remit, and we expect AI-assisted development to be embedded in how we work Apply AI tools to accelerate pipeline development, automated testing, anomaly detection and data quality monitoring Stay current with AI developments in the data engineering space … this is non‐negotiable; you must have built and operated production data pipelines on this stack Strong Python skills – used daily for pipeline development, data transformation and automation Experience across at least two of the three core data domains: Customer, Commercial and Operations ...

Senior Data Engineer Who we are We are an advanced data and business consultancy headquartered in Edinburgh, UK. We are a practitioner-led organisation that collaborates with top consumer brands to drive transformation and foster customer-centricity through our expertise in customer strategy, innovative design … business challenges in an inclusive environment that fosters continuous learning, knowledge sharing, and clear career development. What you will be doing Data Pipeline Development: Design and implement robust, scalable, and efficient data pipelines to collect, transform, and integrate data from various sources. Data ...

advisor to the business and support sound decision-making through data-driven insights and business acumen Work directly with business stakeholders to understand challenges and design complex reports using data from one or multiple systems Deliver reports in formats that support stakeholders, including executive staff … Governance Expertise Experience Presenting to Director and Above Audiences ATS Optimization Keywords Hard Skills Data Modeling Dashboard Design Data Pipeline Design Data Quality Management Data Analysis KPI Knowledge Report Design Data Presentation Conflict Resolution Performance Optimization Soft Skills ...

insurance domain processes, including policy, claims, underwriting, pricing, risk, and regulatory reporting. Hands-on expertise with: Azure Data Factory (ADF) for pipeline orchestration Databricks (PySpark, Delta Lake, MLflow) for data engineering and ML integration Azure ecosystem (Key Vault, Data Lake, Synapse, Functions, Logic … guardrails, governance frameworks, and engineering best practices. Deep technical knowledge of data architecture, integration patterns, CI/CD for data, pipeline monitoring, and data quality frameworks. Excellent stakeholder management and communication skills, particularly with Data Science teams and business leaders. Strong ...

Director of Data and AI Department: Platform Employment Type: Permanent Location: London Description We’re seeking a strategic Director of Data and AI to lead, develop, and empower our Data and AI teams. This role balances leadership with technical excellence; you’ll define … data, analytics, machine learning, and generative AI enable the wider business. You’ll work closely with senior stakeholders, translating business objectives into a clear Data and AI strategy, and ensuring the team delivers scalable, trusted, secure, and commercially impactful solutions. The ultimate goal will ...

Senior Data Engineer at Unisys - Milton Keynes Are you an experienced data engineer with a passion for Generative AI and AI/LM technologies? Unisys is seeking a talented Senior Data Engineer to join our team in Milton Keynes, Buckinghamshire. In this role … science research and trends. Attend a meeting with client stakeholders to discuss their business requirements. Design and implement a data pipeline for processing large datasets. Develop and execute a machine learning model to solve a business problem. Present your findings and recommendations to a variety ...

seeking a Data Engineering Specialist to join the Architecture & Modernization team. You will be instrumental in building … maintaining the data infrastructure for our Data AI platforms. This role will involve hands-on development, data pipeline creation, and close collaboration with stakeholders across the organization. This role requires a self-starter with strong execution skills and the ability to work independently. ...

WHAT YOU'LL BE DOING: Technical Leadership: Own and drive the technical design and architecture of data pipelines, data models, and platform components across the team. Set and enforce engineering standards: code quality, testing, observability, security, and documentation, ensuring the whole team operates to them Lead … Data Product, and wider Tech teams to ensure joined up platform delivery. Maintain accurate and comprehensive technical documentation: architecture decision records, runbooks, pipeline specs, and data dictionaries. Uphold data governance, security, and compliance standards across all data engineering activities. Strong experience ...

NASDAQ: EXLS) is a global data and AI company that offers services and solutions to reinvent client business models, drive better outcomes and unlock growth with speed. Bringing together domain expertise with robust data, powerful analytics, cloud and AI to create agile, scalable solutions and execute … engineering and integration , ideally in enterprise‐scale environments. Hands‐on experience with Spark (PySpark/SparkSQL) , ETL/ELT design, and data pipeline orchestration . Strong data modelling skills (Dimensional, ODS, Data Vault) and experience with data warehousing concepts. Experience working ...

# AWS Data Solution ArchitectBirmingham, Bristol, Newcastle Bank House, Manchester, LondonApply for this job* Permanent* Experienced Professionals* Architecture* ID 466372-en\_GB## **Who You'll Be Working With**As an AWS Data Solution Architect at Capgemini, you will design and lead the delivery of modern data platforms that enable analytics, AI, GenAI, and data driven decision making at scale. You will translate business strategy into robust, governed, and AI ready architectures, taking end to end ownership from vision through to delivery and value realisationYou will be part of the Data ...

Azure Data Stack, critically ADF and Synapse (experience with Microsoft Fabric is a plus) Highly developed python and data pipeline development knowledge, must include substantial PySpark experience Demonstrable DevOps and DataOps experience with an understanding of best practices for engineering, test and ongoing service delivery … understanding of Infrastructure as Code concepts (Demonstrable Terraform experience a plus) Demonstrable experience of Data Pipeline testing, including automated testing, data validation and code assurance Demonstrable experience of working within Agile Delivery projects An understanding of data formats for ingest, transformation and analytics ...

delivery with project management and business development. As a guide, we anticipate the weighting across these will be approximately: Hands‐on data pipeline management and delivery – 50% Project management – 30% Data Pipeline Management and Delivery Designing, building, maintaining and optimising data pipelines … where appropriate). Proposing and implementing enhancements to deliver continuous improvement across our services and operational systems. Facilitating release testing prior to system or pipeline change deployment. Project Management and Delivery Managing the delivery of dataand research projects, ensuring they are delivered accurately, on time and to quality expectations. ...

Data Engineer We are looking for an experienced Data Engineer to join a major data transformation programme on an initial 3-month contract. This is a hands-on role for someone who enjoys building robust, scalable data platforms and pipelines. You will … work as part of a wider technical delivery team, helping to develop a modern AWS-based data environment that supports enterprise reporting, analytics and operational decision-making. The successful candidate will ideally be able to attend the Leeds office one day per week. However, flexible arrangements ...

automation in data engineering — this is a small team that needs to move fast, and we expect the use of AI‐assisted pipeline development and management to be embedded in how we work Data Governance Lead the Data Governance function from a standing … alongside a technology partner or platform team (similar to our Ingenuity relationship) Demonstrated use of AI in data engineering contexts — AI‐assisted pipeline development, automated quality monitoring, or similar Strong leadership and stakeholder management skills — able to influence at Director and C‐suite level Experience ...