1 to 25 of 84 Data Pipeline Jobs in the City of London

Lead Data Engineer

Location
City Of London, England, United Kingdom
Engineering. Engineer governed, trusted data products Build data quality checks, lineage, cataloguing and data observability into every pipeline and data product from the start, rather than adding them after delivery. Implement data contracts, metadata and certification standards … Tables and Snowpark Azure, with Databricks exposure an advantage Advanced SQL and Python Data transformation frameworks such as dbt Data pipeline orchestration, Git-based CI/CD and DataOps Streaming, CDC and event-driven integration API, data sharing and consumption patterns Performance tuning ...

Data Engineering Lead

Location
City Of London, England, United Kingdom
data-enabled products. Establish and champion engineering best practices, including CI/CD, automated testing, version control, monitoring and alerting on pipeline failures. Work closely with Data Architects to implement target-state data models and modern data engineering principles. Partner with … Factory Azure Synapse Analytics Microsoft Fabric Azure Data Lake Advanced SQL development skills. Proficiency in Python and modern data pipeline orchestration tools, including their monitoring and automation. Track record of building reliable, monitored production-grade data pipelines. Experience integrating data ...

Director, OTCR, Data Scientist

Location
City Of London, England, United Kingdom
Model Validation Transformation Programme Support Other Skills Spark/SAS and Large-Scale Data Processing Power BI/Tableau Data Pipeline Development Scenario Analysis and Stress Testing Cross-Functional Collaboration Qualifications 5–8 years of experience in data science, advanced analytics or statistical … modelling, preferably within financial services or risk management. Practical experience in data analysis, machine learning, data pipeline development and data visualisation, with the ability to communicate technical findings to non-technical stakeholders. Experience supporting large-scale transformation or regulatory programmes is desirable. Bachelor ...

Data Engineer – Insurance Domain

Location
City Of London, England, United Kingdom
About Billigence: Billigence is a boutique data consultancy with global outreach and clientele, transforming the way organisations work with data. We leverage proven, cutting-edge technologies to design, tailor, and implement advanced Business Intelligence solutions across Cloud Data Warehousing, Visualisation, Data Science, Data Engineering, and Data Governance. Headquartered in Sydney, Australia, with offices around the world, we help clients navigate complex business challenges, eliminate inefficiencies, and enable scalable, data-driven decision making. About the Role: We are looking for two experienced Data Engineers to join ...

Lead Data Engineer

Location
City Of London, England, United Kingdom
Lead Data Engineer or architect at Made Tech, you'll play a pivotal role in helping public sector organisations become truly data‐lead, by equipping them with robust data platforms. You'll also join a data team on its mission … data knowledge and skills out of silos and embedded into delivery teams. You'll also help implement efficient data pipelines & storage. Key responsibilities Define, shape and perfect data strategies in central and local government. Help public sector teams understand the value of their data ...

SAS Engineer

Hiring Organisation
Hackajob Ltd
Location
City of London, South East, United Kingdom
Employment Type
Permanent Contract, Remote
designs, develops, and operates critical digital environments that drive performance, resilience and sovereignty, helping public and private organizations worldwide retain control over their data and infrastructures, while meeting regulatory requirements. With more than 54,000 employees serving over 4,500 clients across 54 countries, our client helps modernize … Without Strong experience in SAS engineering , development, and administration. Proven expertise in ETL development , data integration, and data pipeline implementation. Experience with data migration, validation, reconciliation, and data quality processes . Scripting experience and the ability to automate migration and transformation ...

Integration Architect

Hiring Organisation
Compass UK & Ireland
Location
City of London, South East, United Kingdom
Employment Type
Permanent Contract, Hybrid
Produce detailed, technically credible integration architectures across all integration domains - SAP and ERP connectivity, e-commerce and digital channels, mobile platforms, data pipeline and analytics integrations, and third-party SaaS and partner systems. Lead technical spikes and proof-of-concept work to validate architectural approaches, evaluate tooling … formerly CPI), IDocs, BAPIs, and the challenges of connecting SAP to modern digital and data platforms. Strong background in data pipeline integration - including CDC, event-to-data flows, real-time feeds, and the interface between integration architecture and data platform design. ...

Data Engineer (Newcastle upon Tyne)

Hiring Organisation
Catalyst
Location
City of London, South East, United Kingdom
Employment Type
Permanent Contract, On-site
fields of management consulting and investment fund management. Based in Newcastle upon Tyne at their enviable facilities, my client seeks a Data Engineer to join their elite Tyneside team as part of rapid expansion of the business to cope with client demand and continued excellent performance. About … engineering best practices. Architecture : Participate in architecture and technical design discussions. Testing: Write automated unit, integration, and end-to-end tests. Data Pipeline Development : Design and develop sophisticated data pipelines to import data from internal and external sources and seamlessly integrate into data ...

BI Engineer

Location
City Of London, England, United Kingdom
seeking a highly technical Business Intelligence Engineer to join our growing BI team. Sitting at the crucial intersection of Data Engineering and Business Intelligence, you will be the technical cornerstone of our BI function. You will architect robust data solutions, automate data ingestion, quality … governance checks, build user-friendly dashboards, and promote a culture of data-driven decision-making across the organization. You will be a self-starter, taking the initiative to understand the whole data lifecycle, Trayport's products and business to transform complex data into actionable ...

Data Engineer - DV Cleared

Location
City Of London, England, United Kingdom
Title: Data Engineer with AI knowledge - Associate Manager Salary: Competitive salary and package (Depending on level of experience) Locations: London, Manchester, Birmingham or Bristol (must be willing to travel to client sites throughout the UK on an ad hoc basis) Please Note: To be eligible for this role … Python Extensive experience with cloud platforms (AWS, GCP, or Azure) Experience with: o Data warehousing and lake architectures o ETL/ELT pipeline development o SQL and NoSQL databases o Distributed computing frameworks (Spark, Kinesis etc) o Software development best practices including CI/ ...

Data Engineer (SC Cleared) - Core Cloud Databricks

Hiring Organisation
Sanderson Government and Defence
Location
City of London, South East, United Kingdom
Employment Type
Temporary Contract, Hybrid
Salary
£569 - £570 per day
environment. This large-scale initiative will require technical expertise across multiple aspects of product migration, alongside the development and testing of new pipeline technologies through Alpha and Beta phases. You will play a key role in designing and implementing an Alpha deployment of Databricks within the new AWS environment … helping to establish the foundations for modern, scalable data pipeline technology. Key Responsibilities Design and implement an Alpha deployment of Databricks in a new AWS environment. Provide technical input and support across multiple aspects of the product estate migration. Support the migration of the government body ...

Senior Data Engineer

Location
City Of London, England, United Kingdom
years. Whatever your priorities – work/life balance, career progression, sustainability, volunteering – you’ll find like-minded people driving change at Howden. Senior Data Engineer Position Senior Data Engineer to own and drive forward high-impact data initiatives, providing technical depth across the data engineering function while supporting the development of junior team members. Summary of the Role Howden Group Services is expanding its internal data engineering capability and is looking for an experienced Senior Data Engineer with strong expertise in Databricks and Azure data services. ...

Lead Data Engineer

Location
City Of London, England, United Kingdom
Whatever your priorities – work/life balance, career progression, sustainability, volunteering – you’ll find like‐minded people driving change at Howden. Position Lead Data Engineer Position Senior Data Engineer to lead and drive forward complex and high‐impact data initiatives. Summary of the Role … Howden Group Services is expanding its internal data engineering capability and is looking for a highly experienced Senior Data Engineer with deep expertise in Databricks and Azure data services. The ideal candidate will have a strong background in designing and delivering scalable, metadata‐driven ...

Data Engineer - Google Ecosystem

Location
City Of London, England, United Kingdom
Data Engineer - London - Hybrid (Monday-Thursday in the office, Friday WFH) - Up to £60,000 DoE - J13194 Currently working with the UK's largest independent media agency to recruit an experienced Data Engineer to join their award-winning Insight, Analytics & Data team. This … exciting opportunity to work with one of the industry's most respected agencies, helping to build cutting-edge cloud-based data solutions that enable clients to maximise the value of their marketing and customer data. You'll play a key role within a specialist Data & Tech ...

Data Engineering Lead

Location
City Of London, England, United Kingdom
build and lead the data engineering capability for 5 Streaming in-house. You’ll own the pipelines, data models and quality controls that join streaming, advertising, marketing and registration data into a single trusted view of content and audience performance. What you will … doing? Own the design, build and operation of the data pipelines and data models that power all streaming reporting, dashboards and analysis. Establish a single joined view of content and audience performance, by creating a common identity and content spine across behavioural, advertising, marketing and registration ...

Geospatial Data Engineer

Hiring Organisation
Coburg Banks Limited
Location
City of London, South East, United Kingdom
Employment Type
Permanent Contract, Hybrid
Salary
£90,000
Role: Geospatial Data Engineer Location: Central London - Hybrid (2-3 days per week in the office) Salary: £75,000 - £90,000 An exciting opportunity for an experienced Geospatial Data Engineer to join an innovative technology business developing automated solutions within the transport and mobility sector. This … hands-on role where you'll play a key part in building accurate, reliable transport datasets and developing the data pipelines and spatial models that underpin software solutions. Working closely with the development team, you'll help transform complex and fragmented data into high-quality, structured ...

Data QA - Test Engineer

Location
City Of London, England, United Kingdom
Role Overview We are partnering with an established investment firm undertaking a core data migration from eFront Invest FIA into an Azure Databricks Lakehouse . We are seeking … hands-on Senior Data QA/Data Quality Engineer to lead automated migration testing, source-to-target reconciliation, and pipeline validation. You will be responsible for ensuring that all data flowing from eFront lands in Databricks with 100% mathematical and schema integrity. ...

Azure Data Support Engineer

Location
City Of London, England, United Kingdom
Azure Data Support Engineer Location: Newcastle or London preferred. Job Type: Full-time | Permanent (On-Call Requirement) Security Clearance Requirement: This role is suitable for candidates who already hold UK Government Security Clearance (SC), or who are eligible and willing to undergo the SC vetting process (eligibility criteria … apply). Key Responsibilities Azure Platform Support & Monitoring Support and maintain Azure-based data solutions, including Azure Data Factory pipelines, datasets, linked services, triggers; Azure Databricks (Spark jobs, notebooks, clusters); Azure Machine Learning models and endpoints; Power BI dashboards and dataset refreshes. Monitor and troubleshoot failures ...

Lead Data Engineer - London

Location
City Of London, England, United Kingdom
Lead Data Engineer - London London | Hybrid - Monday to Thursday in the office. Friday work from home Salary | £85,000-£95,000 dependent on experience Ref J13147 Build the function. Lead the team. Stay hands on. Join a fast-growing AI SaaS business where Data Engineering sits … heart of the platform. This is an opportunity to join at a pivotal stage of the company's growth. As Lead Data Engineer, you'll shape and scale the Data Engineering function, define engineering standards and build a high-performing team, while remaining hands on with ...

Senior Data Engineer

Location
City Of London, England, United Kingdom
About Unifeye Unifeye is a delivery-led, Databricks first consultancy helping organisations unlock the full value of their data and AI. We're a lean, high growth team based in London, working at the intersection of data, technology and business strategy. We partner with enterprise organisations … across industries to solve complex data challenges through modern Lakehouse architectures on Databricks. Joining Unifeye means more than delivering projects. From day one, you'll be trusted to take ownership, challenge conventional thinking and make a genuine impact. With a high degree of autonomy, you'll help shape ...

Lead Data Engineer

Hiring Organisation
Sanderson Recruitment
Location
City of London, South East, United Kingdom
Employment Type
Temporary Contract, Hybrid
Title: Lead Data Engineer Location: London/Hybrid Position Type: 6 Month Contract - Outside IR35 About We're working with a major financial services organisation undergoing a large-scale data transformation programme. They are looking for a hands-on Lead Data Engineer to help … build and scale an enterprise data platform, working closely with senior architects and engineering teams to deliver modern data capabilities across a complex environment. Key Responsibilities Lead the design and build of scalable data engineering solutions using Snowflake and dbt. * Develop data ...

Data Science Associate Manager

Location
City Of London, England, United Kingdom
Role & Details Data Science Associate Manager (Customer) Team: Song AI & Data – AI & Modelling Craft, UK&I Location: London/Manchester Career Level: Associate Manager (L8) We Are Accenture Song accelerates growth and value for our clients through sustained customer relevance. Our capabilities span ideation to execution … Visit us at: https://www.accenture.com/gb-en/about/accenture-song-index The Team Within Accenture Song sits AI & Data, the practice that builds the data-led intelligence behind the customer work. Song AI & Data helps organisations unlock value from ...

Agentic Strategy & Advisory Manager

Location
City Of London, England, United Kingdom
drive growth. We focus on: Reinventing customer service across physical, digital and AI-powered channels Designing future‐ready service models, enabled by platforms, data and automation Embedding GenAI, intelligent automation and decisioning into customer service journeys and operations Supporting clients to define a customer‐first mindset, helping them … Hands‐on experience across visualisation tools, SQL, coding languages and ETL processes, with the ability to engage credibly across the full data pipeline What’s in it for you Our Total Rewards consist of a competitive basic salary, annual performance bonus, opportunities to acquire equity ...

Public Sector Data Platform Architect | Cloud & Pipelines

Location
City Of London, England, United Kingdom
technology consultancy in London is seeking a Lead Data Engineer to help public sector organizations become data-led. This role involves defining data strategies … advising teams on data value, and contributing to recruitment and management efforts. Ideal candidates will possess strong skills in data pipeline creation, architecture design, and have a thorough understanding of agile practices. The role also offers flexible remote working arrangements. #J-18808-Ljbffr ...

Lead Solution Architect (SAM / BEEDS)

Hiring Organisation
Inspire People
Location
City of London, South East, United Kingdom
Employment Type
Temporary Contract, Hybrid
Salary
£90,360
England, in partnership with Inspire People, is recruiting experienced Lead Solution Architects to design and deliver complex, cloud-based solutions across its regulatory data services. Hybrid working from London or Leeds, salary up to £90,360 plus an 8% cash benefits allowance, 10-25% discretionary annual bonus … monetary and financial stability. As part of its ongoing technology modernisation, the Bank is investing in platforms that improve how regulatory data is collected, processed, integrated and used across the organisation. These programmes include modernising the collection of regulatory submissions from banks and insurers, alongside developing internal data ...