1 to 25 of 615 Permanent Data Pipeline Jobs in London

Azure Data Platform Manager Putney, London | Hybrid £85,000+ Bonus + Excellent Benefits | Permanent Azure Data Platform | Azure Architecture | Data Platform Design | Databricks | Data Factory | Data Lake | Synapse Are you an experienced Azure Data Platform Manager, Azure Data Architect or Data Platform Lead with a strong background designing and delivering enterprise-scale Azure data platforms? We're looking for an experienced Azure Data Platform Manager to lead the design, development and optimisation of a modern Azure Data Platform ...

Remote/Hybrid: for London (4 days WFO, 1-day WFH mandatory) Experience 15+ years About the Role We are seeking a senior Data Architect to join the engineering organisation as part of Project Compass. This programme is delivering next-generation capabilities across Accounts (Real-Time Ledger), Payments … Engine, and Foreign Exchange – all of which generate, consume, and depend on high-quality, well-governed data at scale. The Data Architect will own the end-to-end data architecture across spanning Snowflake as the enterprise data warehouse and a landscape ...

engineering principles, big data technologies, cloud computing (specifically Azure), and experience working with large datasets. Key Responsibilities: Data Pipeline Development & Optimisation: o Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various … both technical and non-technical audiences. o Participate in code reviews and knowledge sharing sessions. Automation & DevOps: Implement automation for data pipeline deployments and other data engineering tasks. o Work with DevOps teams to implement and Build CI/CD pipelines, for environmental deployments. ...

Data Engineer – Snowflake We are seeking an experienced Data Engineer with strong expertise in Snowflake, data analysis and data architecture to join our Data Services team and play a key role in delivering our Reporting & Analytics programme. This … exciting opportunity to help shape and develop an enterprise data platform, ensuring high-quality, reliable and accessible data supports business reporting and analytics across the organisation. The Role As a Data Engineer, you will be responsible for designing, developing and maintaining the enterprise data ...

configure and support Microsoft Fabric solutions for our clients across public and private sector environments. This role will support the delivery of modern data platforms, analytics services and integrated data solutions, helping customers make better use of their data through Microsoft Fabric, Power … Support governance and catalogue integration, including alignment with Microsoft Purview where required. Establish monitoring, logging and operational support processes for Fabric workloads. Troubleshoot platform, pipeline, data modelling, refresh, connectivity and performance issues. Support testing, release management and transition into live service. Contribute to technical assurance reviews, identifying ...

direction they choose. About You: You are a motivated and adaptable professional with a strong analytical mindset and a passion for working with data to solve real-world problems. You enjoy working in collaborative, agile teams and take pride in delivering high-quality, data-driven solutions … experience with SQL and at least one programming language such as Python, Scala, or Java. ETL/ELT design and data pipeline orchestration. Streaming and event-driven data architectures. Agile and DevOps delivery methodologies applied to data platforms (DataOps). Expertise in data ...

Olympic Committee, and Mercedes-Benz. Our services span the full digital delivery lifecycle, including architecture, engineering, delivery management, user-centred design, business analysis, data, DevOps, and AI. We operate as a collaborative and inclusive organisation that empowers our people to take ownership, innovate, and develop their expertise. … experience with SQL and at least one programming language such as Python, Scala, or Java. ETL/ELT design and data pipeline orchestration. Streaming and event-driven data architectures. Agile and DevOps delivery methodologies applied to data platforms (DataOps). Expertise in data ...

level proficiency in SQL for complex data transformation, optimization, and analytical workload development Advanced proficiency in Python for data engineering, pipeline automation, and integration development Deep expertise in PySpark for data processing and transformation Proven mastery of cloud platforms with strong preference … Microsoft Azure Expertise in ETL/ELT development and data pipeline orchestration (e.g. Databricks Workflows, DLT, ADF) Extensive experience with Git version control, GitHub collaboration workflows, and modern CI/CD practices including automated testing frameworks and deployment pipeline automation for data engineering workloads ...

Senior Technical Consultant - Data Architect London, UK Job Description Posted Tuesday 25 August 2026 at 00:00 | Expires Wednesday 30 September 2026 at 23:59 Job Role: Data Architect Working Structure: Hybrid (Regular visits to client site) Career Grade: Senior Technical Consultant The role … Data Architect at CACI Information Intelligence Group, you will join a community of dedicated engineers delivering cutting-edge data solutions for our public sector customers across Central Government, Defence, National Security, Critical National Infrastructure and Law Enforcement. This role is focused on architecting and delivering modern ...

Data Architect - Manager Location: London - Hybrid Salary: £80,000- £90,000 We are seeking an experienced AI & Data Architect to join a leading change and transformation consultancy, delivering modern data and analytics solutions across complex client environments. This role will focus … data architecture, cloud data engineering, data governance and AI-enabled data solutions , providing technical leadership across data initiatives. Key Responsibilities as a AI & Data Architect - Manager Design and implement end-to-end data solutions from ingestion ...

Manager - AI & Data Architect £70,000 to 90,000 GBP Bonus Hybrid WORKING Location: Central London, Greater London - United Kingdom Type: Permanent Manager - AI & Data Architect Location: London | Hybrid Salary: Up to £90,000 + bonus We're working with a leading change and transformation consultancy … that is looking to hire an experienced AI & Data Architect into its Technology & Data practice. This is an excellent opportunity for a Data Architect from a consultancy background who enjoys working directly with clients, solving complex business problems and designing modern, scalable data ...

## Lead Product Performance Data Analyst, GlobalApply: London, England: Full time: Posted Today: R24596# **About Vantage**Vantage powers, cools, protects and connects the technology of the world’s well-known hyperscalers, cloud providers and large enterprises. Developing and operating across North America, EMEA and Asia Pacific, Vantage … including Pandas, NumPy or similar libraries.* Experience with ETL/ELT workflows, automated validation routines, controlled data refreshes, or data pipeline documentation.* Exposure to benchmarking, normalization, regional factors, configuration bridges, confidence frameworks, or variance attribution.* Experience working with Developers and IT to translate dataset requirements ...

transformation pipelines (e.g. dbt tests, custom frameworks with Soda). Develop monitoring and observability capabilities to detect anomalies, data drift, and pipeline failures. Implement alerting mechanisms to surface data quality issues to relevant stakeholders in a timely manner. Partner with data engineers … scale datasets. Experience implementing data quality checks and validation frameworks (e.g. dbt tests, Soda, or similar). Familiarity with data pipeline orchestration and transformation tools (e.g. Airflow, Spark, dbt). Experience with cloud-based data platforms, ideally AWS. Understanding of data ...

capital and derivative valuation adjustments. Using the latest analytics and technology such as a fully vectorized pricing library, Machine Learning, and a Big Data stack for scalability, our products and solutions are used by the largest tier-one banks to smaller niche firms. Our risk analytics solutions … London to join the Market Data and Integration team within Risk Analytics. The role will focus on workflow design, data pipeline requirements, integration patterns, validation controls, and operational improvements that support scalable risk analytics delivery. Responsibilities The Senior Product Analyst will work with product, financial ...

ingestion, transformation, and storage processes using tools such as Apache Spark, Azure Data Factory, and other similar technologies Optimize data pipeline performance, ensuring data accuracy, reliability, and timely delivery Mentor and guide junior data engineers, fostering a culture of continuous learning … with at least 3 years in a leadership role Strong proficiency in Apache Spark, Azure Data Factory, and other data pipeline tools Experience with cloud platforms such as Azure, AWS, or GCP Solid understanding of data modeling, data warehousing ...

Principal Data Engineer London (Hybrid) | Contract | Inside IR35 | Up to £650 per day Principal Data Engineer | Financial Services Transformation Programme We are supporting a major financial services transformation programme and are seeking an experienced Principal Data Engineer to play a pivotal role … including Architecture, Product, Engineering, and Business stakeholders. Data Engineering & Platforms Databricks Delta Lake Unity Catalog dbt Spark/PySpark Data pipeline development and optimisation Lakehouse architecture Data integration and ETL/ELT frameworks Data platform design and implementation Cloud Technologies ...

Head of Data & Analytics Department: Technology Reporting to: Chief Technology Officer (CTO) Location: London (Hybrid) Client My client is a rapidly growing UK fintech providing innovative consumer lending solutions through proprietary technology. As an FCA regulated business, data sits at the heart of everything … pipelines Change Data Capture (CDC) AWS Database Migration Service (DMS) Amazon Redshift optimisation SQL performance tuning Data pipeline monitoring Automation Data lifecycle management AWS Data Platform Own the organisation's cloud data platform, including: Amazon Redshift AWS Database Migration ...

it. We pride ourselves on being an organisation of people who genuinely care about helping others. About the Team Data is central to Ofcom's work. We use data from a wide range of sources to understand regulated sectors and provide insight that informs critical policy … operational and regulatory decisions. As Ofcom's responsibilities continue to expand, the volume, complexity and strategic importance of data continues to grow. To support this ambition, Ofcom is investing in a modern cloud-based data platform built on Microsoft Azure and the wider Microsoft data ...

sends, logistics, offers, and reporting, but also the analytical core that makes it work, building the response model and sourcing the data pipeline and user-data enrichment that let us target the right customer, with the right offer, at the right moment. … modelling, automation)**Desirable:*** Experience in financial services, regulated or other consumer categories where offers and eligibility matter* Experience standing up a data pipeline or enrichment stack (first-party + third-party/bureau data)* Experience building a channel or function from scratch* Familiarity with credit ...

Primary Responsibilities Client Engagement & Delivery Data Pipeline Development (Batch and Streaming) Databricks & Lakehouse Architectures Data Modelling & Optimisation (Delta Lake, Medallion architecture) Collaboration & Best Practices Quality, Governance & Security Business Relationships Solution Architects Data Engineers, Developers, ML Engineers, and Analysts Client stakeholders up to Head … Data Engineering, Chief Data Architect, and Analytics leadership Key Competencies Proven experience in data engineering and pipeline development on Databricks and cloud-native platforms. Strong consulting values with ability to collaborate effectively in client-facing environments. Hands‐on expertise across the data ...

world’s largest asset class at over $145 trillion. Debt markets are vast, global, and mission-critical, yet still run on fragmented data, PDFs, and manual workflows. 9fin replaces this broken infrastructure with a single platform that centralises proprietary credit data, deep analysis, and high-value … markets worldwide. The Opportunity 9fin has built one of the most comprehensive datasets in credit and debt markets: news, financials, cap tables, instrument data, legal documents, and ESG analysis. Today, most clients access that data through 9fin.com. But a large and growing segment of the market ...

Senior Data Engineer – Market Data We are looking for a Senior Data Engineer to build and operate a high-performance market data platform supporting quantitative research and production trading systems. You'll design scalable data pipelines, process billions of market … engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience ...

Director of Data and AI Department: Platform Employment Type: Permanent Location: London Description We’re seeking a strategic Director of Data and AI to lead, develop, and empower our Data and AI teams. This role balances leadership with technical excellence; you’ll define … data, analytics, machine learning, and generative AI enable the wider business. You’ll work closely with senior stakeholders, translating business objectives into a clear Data and AI strategy, and ensuring the team delivers scalable, trusted, secure, and commercially impactful solutions. The ultimate goal will ...

# AWS Data Solution ArchitectBirmingham, Bristol, Newcastle Bank House, Manchester, LondonApply for this job* Permanent* Experienced Professionals* Architecture* ID 466372-en\_GB## **Who You'll Be Working With**As an AWS Data Solution Architect at Capgemini, you will design and lead the delivery of modern data platforms that enable analytics, AI, GenAI, and data driven decision making at scale. You will translate business strategy into robust, governed, and AI ready architectures, taking end to end ownership from vision through to delivery and value realisationYou will be part of the Data ...

delivery with project management and business development. As a guide, we anticipate the weighting across these will be approximately: Hands‐on data pipeline management and delivery – 50% Project management – 30% Data Pipeline Management and Delivery Designing, building, maintaining and optimising data pipelines … where appropriate). Proposing and implementing enhancements to deliver continuous improvement across our services and operational systems. Facilitating release testing prior to system or pipeline change deployment. Project Management and Delivery Managing the delivery of dataand research projects, ensuring they are delivered accurately, on time and to quality expectations. ...