251 to 275 of 363 Data Pipeline Jobs in Central London

Up to £125,000 base + bonus & benefits A leading global consulting and analytics firm is seeking a Databricks Architect to drive architecture and delivery of large-scale data solutions for their commercial insurance clients. This is a senior role, shaping data strategy and modernising platforms that underpin underwriting, claims, and pricing. Key Responsibilities Lead end … to-end architecture and solution design across Databricks and Azure cloud ecosystems. Build scalable lakehouse architectures, advanced analytics, and AI/ML data pipelines. Work closely with insurance stakeholders to translate business needs into technical solutions. Ensure best practices in data governance, security, and regulatory compliance. Provide technical leadership to engineering teams and contribute to architecture … standards. Skills & Experience Proven experience as a Data/Solution Architect with strong expertise in Databricks (Delta Lake, MLflow, Spark) . Deep understanding of commercial insurance processes (underwriting, claims, personal/specialty lines). Strong background in Azure (Synapse, Data Factory, Fabric) or similar cloud data platforms. Knowledge of data governance, integration More ❯

scale the platform, and push the limits of cloud security tech. You’ll join a world-class team solving complex engineering challenges - from distributed systems and large-scale data pipelines to real-time detection and automation. Every line of code you write will have real impact. 💡 They’re after someone who: Has solid OOP experience (Python, Java, Go More ❯

enterprise-scale systems. Role & Responsibilities: Lead the design and delivery of enterprise applications and reporting systems Develop tools for dashboards, analytics, and operational monitoring Build and maintain APIs, data pipelines, and automation for business processes Implement scalable solutions to improve operational efficiency and data quality Enhance and integrate commercial off-the-shelf (COTS) platforms where required … automation, and Agile delivery Knowledge of compliance, regulatory reporting, and enterprise operational processes Proven experience delivering complex software solutions Strong leadership and stakeholder management skills Bonus: Python, Azure Data Factory/Databricks, data analysis, and COTS platforms Package: £85,000 15% non-contributory pension Hybrid working (3 days per week) Performance-based bonus scheme Comprehensive benefits More ❯

a Technical Business Analyst with deep knowledge of insurance pricing and underwriting systems to bridge the gap between actuarial, product and engineering teams. You’ll gather requirements, design data flows, and help implement pricing models, APIs and analytical capabilities that drive better underwriting decisions. Key Responsibilities: Define and document requirements for pricing system integrations, model deployment and API … workflows. Translate pricing models (Excel/Python/R) into technical specifications for developers and data engineers. Collaborate with underwriters and actuaries to improve model governance, version control and reporting. Support UAT, model validation and release testing. Deliver insights and metrics around model performance, quote success and underwriting efficiency. Partner with insurers and MGAs to streamline pricing and … a Business Analyst or Technical Analyst role within insurance, reinsurance or InsurTech. Experience with pricing or rating systems, underwriting workflows or model integration projects. Strong understanding of insurance data structures, rating factors and model governance. Skilled in SQL and BI tools (Power BI, Tableau, Looker). Comfortable working with APIs, data pipelines and collaborating with developers. More ❯

a Technical Business Analyst with deep knowledge of insurance pricing and underwriting systems to bridge the gap between actuarial, product and engineering teams. You’ll gather requirements, design data flows, and help implement pricing models, APIs and analytical capabilities that drive better underwriting decisions. Key Responsibilities: Define and document requirements for pricing system integrations, model deployment and API … workflows. Translate pricing models (Excel/Python/R) into technical specifications for developers and data engineers. Collaborate with underwriters and actuaries to improve model governance, version control and reporting. Support UAT, model validation and release testing. Deliver insights and metrics around model performance, quote success and underwriting efficiency. Partner with insurers and MGAs to streamline pricing and … a Business Analyst or Technical Analyst role within insurance, reinsurance or InsurTech. Experience with pricing or rating systems, underwriting workflows or model integration projects. Strong understanding of insurance data structures, rating factors and model governance. Skilled in SQL and BI tools (Power BI, Tableau, Looker). Comfortable working with APIs, data pipelines and collaborating with developers. More ❯

Multiple Data Engineers/Scientists/Engineers Needed 📍 Location: London - Remote First but there will be travel to client sites and onboarding will be done in London. 💸 Salary: £55,000 – £100,000 + Bonus + Equity Options 🛡️ Security Cleared or Clearable Candidates Only Areti Group is working with one of the UK's fastest-growing, well-funded tech … start-ups, is hiring Multiple Data Engineers/Data Scientists/ML Analysts to join our cutting-edge team in London. We're on a mission to scale rapidly – with plans to grow by 40 heads this year – and this is your chance to be a part of something big from the ground up. We are … working on high-impact, mission-critical projects across the public sector, defence, and government organisations , delivering real-world innovation powered by data and technology . 🔧 Tech Stack & Skills We're Looking For: Palantir Azure Databricks Microsoft Azure Python Docker & Kubernetes Linux Apache Tools Data Pipelines IoT (Internet of Things) Scrum/Agile Methodologies ✅ Ideal Candidate: Already More ❯

Multiple Data Engineers/Scientists/Engineers Needed 📍 Location: London - Remote First but there will be travel to client sites and onboarding will be done in London. 💸 Salary: £55,000 – £100,000 + Bonus + Equity Options 🛡️ Security Cleared or Clearable Candidates Only Areti Group is working with one of the UK's fastest-growing, well-funded tech … start-ups, is hiring Multiple Data Engineers/Data Scientists/ML Analysts to join our cutting-edge team in London. We're on a mission to scale rapidly – with plans to grow by 40 heads this year – and this is your chance to be a part of something big from the ground up. We are … working on high-impact, mission-critical projects across the public sector, defence, and government organisations , delivering real-world innovation powered by data and technology . 🔧 Tech Stack & Skills We're Looking For: Palantir Azure Databricks Microsoft Azure Python Docker & Kubernetes Linux Apache Tools Data Pipelines IoT (Internet of Things) Scrum/Agile Methodologies ✅ Ideal Candidate: Already More ❯

Job Title: Data Scientist Salary: £65,000-£75,000 + Benefits Location: West London - Hybrid (3 days p/w in-office) Tech: AWS, Snowflake, Airflow, DBT The Company: Immersum are supporting the continued growth of a leading PropTech company on a mission to revolutionise how the property sector understands people, places, and data. By combining cutting-edge … data science with powerful location intelligence, they help major organisations make smarter, faster decisions. Backed by top-tier investors and growing fast, this is your chance to shape the future of PropTech from the inside. The Role Requirements: As a Senior Data Scientist, you’ll develop, deploy, and optimise data science solutions that sit … at the heart of the company’s product offerings. You’ll play a critical role in shaping how data-driven insights fuel smarter decisions across the business. What you’ll be doing: Spotting opportunities to apply data science and ML to solve meaningful product challenges Designing and building data pipelines to support experimentation and More ❯

Job Title: Data Scientist Salary: £65,000-£75,000 + Benefits Location: West London - Hybrid (3 days p/w in-office) Tech: AWS, Snowflake, Airflow, DBT The Company: Immersum are supporting the continued growth of a leading PropTech company on a mission to revolutionise how the property sector understands people, places, and data. By combining cutting-edge … data science with powerful location intelligence, they help major organisations make smarter, faster decisions. Backed by top-tier investors and growing fast, this is your chance to shape the future of PropTech from the inside. The Role Requirements: As a Senior Data Scientist, you’ll develop, deploy, and optimise data science solutions that sit … at the heart of the company’s product offerings. You’ll play a critical role in shaping how data-driven insights fuel smarter decisions across the business. What you’ll be doing: Spotting opportunities to apply data science and ML to solve meaningful product challenges Designing and building data pipelines to support experimentation and More ❯

Senior Software Engineer (Data-Centric) Manchester or South London (Hybrid – 2 x a week in office) £70,000 – £85,000 + 10% bonus + excellent benefits Our client is a global self-defence intelligence and technology organisation helping governments and defence agencies act with confidence. They use AI, graph databases, cloud, and search technologies to deliver faster, smarter … Develop in one or more core technology areas: NoSQL (MongoDB, DocumentDB) Graph databases (GraphDB, Neo4j) Search technologies (ElasticSearch, OpenSearch, SOLR) ML/AI frameworks (SageMaker, Bedrock) Cloud-based data engineering (AWS) Build and maintain cloud-native data pipelines, ensuring data quality, accuracy, and reliability. What We’re Looking For 5+ years’ experience in software … ability to deliver reliable, scalable systems using cloud technologies (AWS preferred). Strong understanding of system design , API-first development , and SDLC best practices . Experience with complex data systems - structured, unstructured, or graph-based. Curious, proactive, and comfortable “rolling your sleeves up” to solve hard problems. Why Join? Work on advanced data and intelligence platforms More ❯

Senior Software Engineer (Data-Centric) Manchester or South London (Hybrid – 2 x a week in office) £70,000 – £85,000 + 10% bonus + excellent benefits Our client is a global self-defence intelligence and technology organisation helping governments and defence agencies act with confidence. They use AI, graph databases, cloud, and search technologies to deliver faster, smarter … Develop in one or more core technology areas: NoSQL (MongoDB, DocumentDB) Graph databases (GraphDB, Neo4j) Search technologies (ElasticSearch, OpenSearch, SOLR) ML/AI frameworks (SageMaker, Bedrock) Cloud-based data engineering (AWS) Build and maintain cloud-native data pipelines, ensuring data quality, accuracy, and reliability. What We’re Looking For 5+ years’ experience in software … ability to deliver reliable, scalable systems using cloud technologies (AWS preferred). Strong understanding of system design , API-first development , and SDLC best practices . Experience with complex data systems - structured, unstructured, or graph-based. Curious, proactive, and comfortable “rolling your sleeves up” to solve hard problems. Why Join? Work on advanced data and intelligence platforms More ❯

Hiring: Data Engineer | Mission-Driven Fintech | Remote (UK based) 🚀 Propel are proud to be partnering with a rapidly scaling global fintech that’s transforming financial access for immigrants and international communities across the world. 📈 As their lending business continues to grow, they're looking for a Data Engineer to help scale the data infrastructure … that powers smarter, fairer credit decisions. 🧠 If you're passionate about building robust data pipelines, enabling real-time underwriting, and turning complex datasets into production-ready insights — this is your chance to work at the heart of a mission-driven business. ✅ What you'll bring: - 4+ years as a Data Engineer (ideally in fintech/lending … Strong SQL, Python, Snowflake, dbt, Dagster - Experience with credit bureau data, open banking APIs - A love for clean, scalable data systems in fast-moving environments Sound like you or someone you know? 📩 Drop me a message or apply directly – I’d love to tell you more More ❯

Are you a Data Engineer who loves solving real-world data problems and building scalable solutions from the ground up? We’re supporting a well-established business in the digital transformation space, and they’re looking for someone to help drive data platform development across multiple business areas and tech environments. Data Engineer £55,000-£65,000 per annum London based 2 days a week The role: You’ll work closely with technical and non-technical teams, owning data pipelines, improving performance, and getting involved in projects that span cloud, automation, and AI-readiness This role is perfect for someone who enjoys building and optimising data … moving environment, and wants to work across a variety of platforms and challenges. What you'll need: Solid hands-on experience with Python and SQL Strong knowledge of data pipelines, cloud infrastructure (AWS & Azure), and integration tools Familiarity with tools like Airflow and version control, GIT. Confident working with structured and unstructured datasets Comfortable contributing to both technical More ❯

process, with the founders having had first hand experience of the lengthiness and challenges of deal processes in the past. Their stack spans across back end, front end, data and crucially AI. This is a great opportunity for an entrepreneurial software engineer who wants to play a part in shaping the technical vision of this business and work … their product from an early stage. What you'll work on: Backend APIs (Python/FastAPI): Build and maintain secure, high-performance services that drive AI features and data access at scale. RAG & vector search: Design and improve retrieval pipelines (embeddings, chunking, hybrid search, ranking, feedback loops), owning schema design, latency, and relevance across vector databases. LLM integration … Connect and orchestrate large language models (OpenAI, Bedrock, etc.), manage prompts, tools, safeguards, and evaluation. Data pipelines: Ingest, clean, and transform structured and unstructured data; design efficient schemas (Postgres/NoSQL) for search and analytics. Frontend (React/Next.js): Deliver user-friendly, performant UIs that make AI-powered features (search, filters, explanations, citations) clear and accessible. More ❯

process, with the founders having had first hand experience of the lengthiness and challenges of deal processes in the past. Their stack spans across back end, front end, data and crucially AI. This is a great opportunity for an entrepreneurial software engineer who wants to play a part in shaping the technical vision of this business and work … their product from an early stage. What you'll work on: Backend APIs (Python/FastAPI): Build and maintain secure, high-performance services that drive AI features and data access at scale. RAG & vector search: Design and improve retrieval pipelines (embeddings, chunking, hybrid search, ranking, feedback loops), owning schema design, latency, and relevance across vector databases. LLM integration … Connect and orchestrate large language models (OpenAI, Bedrock, etc.), manage prompts, tools, safeguards, and evaluation. Data pipelines: Ingest, clean, and transform structured and unstructured data; design efficient schemas (Postgres/NoSQL) for search and analytics. Frontend (React/Next.js): Deliver user-friendly, performant UIs that make AI-powered features (search, filters, explanations, citations) clear and accessible. More ❯

process, with the founders having had first hand experience of the lengthiness and challenges of deal processes in the past. Their stack spans across back end, front end, data and crucially AI. This is a great opportunity for an entrepreneurial software engineer who wants to play a part in shaping the technical vision of this business and work … their product from an early stage. What you'll work on: Backend APIs (Python/FastAPI): Build and maintain secure, high-performance services that drive AI features and data access at scale. RAG & vector search: Design and improve retrieval pipelines (embeddings, chunking, hybrid search, ranking, feedback loops), owning schema design, latency, and relevance across vector databases. LLM integration … Connect and orchestrate large language models (OpenAI, Bedrock, etc.), manage prompts, tools, safeguards, and evaluation. Data pipelines: Ingest, clean, and transform structured and unstructured data; design efficient schemas (Postgres/NoSQL) for search and analytics. Frontend (React/Next.js): Deliver user-friendly, performant UIs that make AI-powered features (search, filters, explanations, citations) clear and accessible. More ❯

You’ll be joining an established global commodities trading firm where world-class data analysts are already building sophisticated, high-impact models. They now need the tech muscle to take them to production. In this crucial role, you’ll connect sophisticated analytics with scalable infrastructure, manage CI/CD pipelines, containerized systems, and AWS environments that underpin the … firm’s data lake, data pipelines, and analytics platforms. You’ll build robust infrastructure, automate data ingestion, and deploy cutting-edge analytical models into live trading environments. Working alongside brilliant analysts and engineers, you’ll shape the firm’s technical foundation and directly influence how data drives trading strategy and decision-making. More ❯

You’ll be joining an established global commodities trading firm where world-class data analysts are already building sophisticated, high-impact models. They now need the tech muscle to take them to production. In this crucial role, you’ll connect sophisticated analytics with scalable infrastructure, manage CI/CD pipelines, containerized systems, and AWS environments that underpin the … firm’s data lake, data pipelines, and analytics platforms. You’ll build robust infrastructure, automate data ingestion, and deploy cutting-edge analytical models into live trading environments. Working alongside brilliant analysts and engineers, you’ll shape the firm’s technical foundation and directly influence how data drives trading strategy and decision-making. More ❯

Senior Data Engineer (GenAI) - Leading Tech Company | London (Hybrid) Hybrid: 1 day in the office/4 days WFH Permanent role only Represented by Robert Half Limited Please note: No visa sponsorship on offer, No B2B,B2C, Fully remote workers or contractors on offer. The Opportunity We're looking for a Senior Data Engineer with hands … on GenAI experience to join a forward-thinking technology business that's building data-driven solutions with real-world impact. This is a highly visible, senior-level role where you'll take ownership of data pipelines, architecture, and GenAI operational frameworks - helping shape the next generation of AI-driven products and analytics. If you enjoy combining … engineering rigor with AI innovation, this role gives you the platform, freedom, and backing to make it happen. Key Responsibilities Design, build, and maintain scalable data pipelines across modern cloud platforms Drive the implementation of GenAI frameworks, including model deployment, monitoring, and lifecycle management ('GenAI Ops') Develop robust ETL/ELT workflows and optimise performance within Snowflake Collaborate More ❯

Lead Data Engineer - Python, SQL, GCP, Kubernetes, DBT, Airflow and BigQurey OB have partnered with a leading scale-up business in the AdTech space, where Data is at the forefront of everything they do, and they are currently hiring for a Lead Data Engineer to join their team and work on the development of … their Data platform. This will be a hands-on role but you will line manage a small team of 3 Engineers, where the long-term plan of this team is to grow and this is a great opportunity to progress in a scaling business. This role requires candidates to come with hands on experience working in Data heavy environments, with Real-Time Data Pipelines, Distributed Streaming Pipelines and strong knowledge of Cloud Environments. Lead Data Engineer - Python, SQL, GCP, Kubernetes, DBT, Airflow and BigQurey Key Skills and Experience: Python, SQL, GCP, Kubernetes, DBT, Airflow and BigQurey Prior experience working as a Lead Engineer or Tech Lead Pays £100k-£130k + bonus More ❯

We’re partnering with a high-growth AI startup that’s building intelligent automation systems designed to simplify complex, real-world operations. Their platform is already redefining how data, workflow, and human decision-making come together — and they’re scaling fast. If you’re excited by the idea of joining an elite engineering team tackling deep technical problems … intelligent features into production — solving high-impact problems with clean, scalable code. Responsibilities: Architect, design, and build robust backend systems and APIs using Python. Collaborate with AI and data teams to integrate models into production environments. Optimise and scale distributed systems and data pipelines for performance and reliability. Contribute to overall technical strategy, code reviews, and … control, CI/CD workflows, and containerised development. Excellent communication and problem-solving skills. Bonus Points For: Experience in AI-driven or automation-heavy environments. Exposure to PropTech, data-heavy SaaS platforms, or real-world workflow automation. Familiarity with microservices and orchestration tools (Docker, Kubernetes). What’s on Offer: Competitive salary + equity options. The opportunity to More ❯

We’re partnering with a high-growth AI startup that’s building intelligent automation systems designed to simplify complex, real-world operations. Their platform is already redefining how data, workflow, and human decision-making come together — and they’re scaling fast. If you’re excited by the idea of joining an elite engineering team tackling deep technical problems … intelligent features into production — solving high-impact problems with clean, scalable code. Responsibilities: Architect, design, and build robust backend systems and APIs using Python. Collaborate with AI and data teams to integrate models into production environments. Optimise and scale distributed systems and data pipelines for performance and reliability. Contribute to overall technical strategy, code reviews, and … control, CI/CD workflows, and containerised development. Excellent communication and problem-solving skills. Bonus Points For: Experience in AI-driven or automation-heavy environments. Exposure to PropTech, data-heavy SaaS platforms, or real-world workflow automation. Familiarity with microservices and orchestration tools (Docker, Kubernetes). What’s on Offer: Competitive salary + equity options. The opportunity to More ❯

of 12 months ending December 2024 totaled $13.8 billion. Experience : Minimum 10+ Years Strong Knowledge in Hadoop, Kafka, SQL/NoSQL Specialization in designing and implementing large-scale data pipelines, ETL processes, and distributed systems Should be able to work independenty with minimal help/guidance Good Understanding of Airflow,Data Fusion and Data Flow Strong Background and experience in Data Ingestions,Transformation,Modeling and Performance tuning. One migration Experience from Cornerstone to GCP will be added advantage Suppport the design and development of BigData echosystem Experience in building complex SQL Queries Strong Communication Skills More ❯

to shape the next generation of AI infrastructure. What You’ll Do Design, implement, and optimize LangGraph-based AI workflows and multi-agent systems Integrate LLMs, APIs, and data pipelines into production-ready solutions Build scalable, maintainable AI applications with structured memory, tools, and dynamic reasoning paths Collaborate with cross-functional teams — from data scientists to … Creative problem-solver with a bias toward action and experimentation Nice to Have Experience deploying AI agents at scale Knowledge of cloud platforms (AWS, GCP, Azure) Background in data engineering, NLP, or ML Ops Why Join Us Work on cutting-edge AI projects with real-world impact Collaborate with a world-class engineering team Flexible remote environment with More ❯

to shape the next generation of AI infrastructure. What You’ll Do Design, implement, and optimize LangGraph-based AI workflows and multi-agent systems Integrate LLMs, APIs, and data pipelines into production-ready solutions Build scalable, maintainable AI applications with structured memory, tools, and dynamic reasoning paths Collaborate with cross-functional teams — from data scientists to … Creative problem-solver with a bias toward action and experimentation Nice to Have Experience deploying AI agents at scale Knowledge of cloud platforms (AWS, GCP, Azure) Background in data engineering, NLP, or ML Ops Why Join Us Work on cutting-edge AI projects with real-world impact Collaborate with a world-class engineering team Flexible remote environment with More ❯