226 to 250 of 302 Data Engineering Jobs in the City of London

organizations unlock the value of technology and build a more sustainable, more inclusive world. Your Role We are looking for a skilled Spark/Scala Developer to join our data engineering team. The ideal candidate will have hands-on experience in designing, developing, and maintaining large-scale data processing pipelines using Apache Spark and Scala. You will … work closely with data scientists, analysts, and engineers to build efficient data solutions and enable data-driven decision-making. Key Responsibilities: Develop, optimize, and maintain data pipelines and ETL processes using Apache Spark and Scala. Design scalable and robust data processing solutions for batch and real-time data. Collaborate with cross-functional teams to gather … requirements and translate them into technical specifications. Perform data ingestion, transformation, and cleansing from various structured and unstructured sources. Monitor and troubleshoot Spark jobs, ensuring high performance and reliability. Write clean, maintainable, and well-documented code. Participate in code reviews, design discussions, and agile ceremonies. Implement data quality and governance best practices. Stay updated with emerging technologies and More ❯

organizations unlock the value of technology and build a more sustainable, more inclusive world. Your Role We are looking for a skilled Spark/Scala Developer to join our data engineering team. The ideal candidate will have hands-on experience in designing, developing, and maintaining large-scale data processing pipelines using Apache Spark and Scala. You will … work closely with data scientists, analysts, and engineers to build efficient data solutions and enable data-driven decision-making. Key Responsibilities: Develop, optimize, and maintain data pipelines and ETL processes using Apache Spark and Scala. Design scalable and robust data processing solutions for batch and real-time data. Collaborate with cross-functional teams to gather … requirements and translate them into technical specifications. Perform data ingestion, transformation, and cleansing from various structured and unstructured sources. Monitor and troubleshoot Spark jobs, ensuring high performance and reliability. Write clean, maintainable, and well-documented code. Participate in code reviews, design discussions, and agile ceremonies. Implement data quality and governance best practices. Stay updated with emerging technologies and More ❯

share options , ensuring youre rewarded as the business grows. The Role This is an exciting opportunity for a Software Engineer looking to make an impact within a high-performance data and analytics environment . Working in the Data Distribution team , youll help design and develop systems that distribute vast amounts of financial data efficiently, securely, and at … scale. Youll play a critical role in developing APIs, workflow automation, and CI/CD pipelines ensuring the reliability, performance, and scalability of the companys data delivery systems. What Youll Do Design and build systems to distribute large datasets to global clients. Develop API and system solutions that follow best engineering practices. Implement workflow management tools to automate … CI/CD pipelines for continuous testing and deployment. Collaborate closely with product and operations teams to deliver seamless solutions. Evaluate and integrate emerging tools and technologies in the data engineering space. What Were Looking For Essential: Degree in Computer Science, Engineering, or a related STEM subject. 2+ years experience as a Software Engineer (or Data More ❯

Data Science Engineer My client is recruiting for a Data Science Engineer to design, develop, and deliver AI and analytics solutions aligned with the organisations Data & AI strategy. Key Responsibilities * End-to-end development of AI/ML solutions. * MLOps practices: CI/CD, model monitoring, retraining. * Use of open-source and enterprise tools (LangChain, Azure OpenAI … Databricks). * Generative AI features: embeddings, RAG, AI agents. * Clean, testable code with modern engineering practices. * Align with enterprise architecture and governance. * Collaborate with architects and stakeholders. * Lifecycle management of models. * Pilot emerging technologies. Experience & Skills * 2-4 years in production-level AI/ML delivery. * Legal/professional services experience is a plus. * AI/ML frameworks: PyTorch … TensorFlow, LangChain. * Cloud: Azure (preferred), AWS, GCP. * MLOps: CI/CD, model lifecycle, monitoring. * Generative AI: LLMs, RAG, chat agents. * Data engineering alignment: ETL, governance. * Strong coding, communication, and collaboration skills. * Strategic thinking, problem-solving, and stakeholder engagement. In accordance with the Employment Agencies and Employment Businesses Regulations 2003, this position is advertised based upon DGH Recruitment Limited More ❯

options , ensuring you’re rewarded as the business grows. The Role This is an exciting opportunity for a Software Engineer looking to make an impact within a high-performance data and analytics environment . Working in the Data Distribution team , you’ll help design and develop systems that distribute vast amounts of financial data efficiently, securely, and … at scale. You’ll play a critical role in developing APIs, workflow automation, and CI/CD pipelines — ensuring the reliability, performance, and scalability of the company’s data delivery systems. What You’ll Do Design and build systems to distribute large datasets to global clients. Develop API and system solutions that follow best engineering practices. Implement workflow … CI/CD pipelines for continuous testing and deployment. Collaborate closely with product and operations teams to deliver seamless solutions. Evaluate and integrate emerging tools and technologies in the data engineering space. What We’re Looking For Essential: Degree in Computer Science, Engineering, or a related STEM subject. 2+ years’ experience as a Software Engineer (or Data More ❯

the world’s leading organizations unlock the value of technology and build a more sustainable, more inclusive world. YOUR ROLE We are seeking a skilled and hands-on AWS Data Engineer with strong coding expertise and deep experience in building scalable data solutions using AWS services. The ideal candidate will have a solid background in data engineering, Python development, and cloud-native architecture. YOUR PROFILE Design, develop, and maintain robust data pipelines and ETL workflows using AWS services. Implement scalable data processing solutions using PySpark and AWS Glue. Build and manage infrastructure as code using CloudFormation. Develop and deploy serverless applications using AWS Lambda, Step Functions, and S3. Perform data querying and analysis … using Athena. Collaborate with Data Scientists to operationalize models using SageMaker. Ensure secure and compliant data handling using IAM, KMS, and VPC configurations. Containerize applications using ECS for scalable deployment. Write clean, testable code in Python, with a strong emphasis on unit testing. Use GitLab for version control, CI/CD, and collaboration. Strong coding background in Python More ❯

Snowflake Centre of Excellence Lead - London Who we are: We deliver powerful data and AI solutions that minimise operational cost, strengthen resilience against risk, and uncover revenue opportunity. Our clients can retain our people to drive lasting adoption while futureproofing their workforce with exceptional talent. We’re the preferred partner of today’s leading technology providers, including Databricks, Snowflake … and Collibra, to accelerate delivery and co-create revolutionary solutions. We transform the lives of our consultants, clients, and their customers through data and AI. Role Purpose: As part of our Core Practice, you will be joining a tight knit team of motivated technology professionals where ongoing learning and development are central to our ethos and internal culture. You … developing Snowflake-focused intellectual property, including best practices, accelerators, reusable frameworks, and reference architectures Strengthen our Snowflake partnership by leveraging technical credibility to build trusted relationships with architecture and engineering stakeholders at Snowflake Provide technical assurance on delivery projects, ensuring that all solutions adhere to established Snowflake implementation best practices Contribute to pre-sales activities by shaping proposals, supporting More ❯

Snowflake Centre of Excellence Lead - London Who we are: We deliver powerful data and AI solutions that minimise operational cost, strengthen resilience against risk, and uncover revenue opportunity. Our clients can retain our people to drive lasting adoption while futureproofing their workforce with exceptional talent. We’re the preferred partner of today’s leading technology providers, including Databricks, Snowflake … and Collibra, to accelerate delivery and co-create revolutionary solutions. We transform the lives of our consultants, clients, and their customers through data and AI. Role Purpose: As part of our Core Practice, you will be joining a tight knit team of motivated technology professionals where ongoing learning and development are central to our ethos and internal culture. You … developing Snowflake-focused intellectual property, including best practices, accelerators, reusable frameworks, and reference architectures Strengthen our Snowflake partnership by leveraging technical credibility to build trusted relationships with architecture and engineering stakeholders at Snowflake Provide technical assurance on delivery projects, ensuring that all solutions adhere to established Snowflake implementation best practices Contribute to pre-sales activities by shaping proposals, supporting More ❯

/day (Inside IR35 – depending on experience) About the Role: We’re looking for an experienced PySpark + Fabric Developer to join a leading global financial markets and data services organisation. You’ll be part of a team driving large-scale data transformation and modernisation projects. Key Responsibilities: Design, build, and optimise scalable data pipelines (batch & streaming … . Develop and manage dataflows and semantic models supporting analytics. Implement robust data validations, cleansing, and performance tuning. Build ETL workflows using modern data engineering best practices. Collaborate with business stakeholders to deliver fit-for-purpose solutions. Ensure compliance with data security, access control, and governance. Skills & Experience: Strong hands-on experience in PySpark (RDDs, DataFrames … Spark SQL). Proven experience in Microsoft Fabric , Data Lake , and ETL pipeline design. Familiarity with time-series data, market feeds, and financial datasets . Experience with Git, CI/CD pipelines, and DevOps practices . Excellent communication skills and the ability to work collaboratively. Nice to Have: Knowledge of Power BI integration and OneLake . Understanding of More ❯

core technology crucial to Ki's success – allowing us to evolve underwriting intelligently and unlock massive scale. We're a multi-disciplined team, bringing together expertise in software and data engineering, full stack development, platform operations, algorithm research, and data science. Our squads focus on delivering high-impact features – we favour a highly iterative, analytical approach. We … re looking for a Staff Software Engineer to join our Engineering team. You will provide technical leadership for both delivery squads and the wider engineering organisation. Along with our engineering leadership, you’ll champion engineering excellence and high standards of quality through an environment of continuous improvement. You will be hands-on, directly delivering new impactful … for growth and improvement - Build mock-ups or prototypes to explore and troubleshoot new initiatives - Explore new ideas and emerging technologies, develop prototypes quickly - Uphold and advance the wider engineering team’s principles and ways of working - Promote a culture of continuous improvement, positively influence current ways of working, leading by example - Serve as a domain expert in many More ❯

core technology crucial to Ki's success – allowing us to evolve underwriting intelligently and unlock massive scale. We're a multi-disciplined team, bringing together expertise in software and data engineering, full stack development, platform operations, algorithm research, and data science. Our squads focus on delivering high-impact features – we favour a highly iterative, analytical approach. We … re looking for a Staff Software Engineer to join our Engineering team. You will provide technical leadership for both delivery squads and the wider engineering organisation. Along with our engineering leadership, you’ll champion engineering excellence and high standards of quality through an environment of continuous improvement. You will be hands-on, directly delivering new impactful … for growth and improvement - Build mock-ups or prototypes to explore and troubleshoot new initiatives - Explore new ideas and emerging technologies, develop prototypes quickly - Uphold and advance the wider engineering team’s principles and ways of working - Promote a culture of continuous improvement, positively influence current ways of working, leading by example - Serve as a domain expert in many More ❯

Mid & Senior Data Engineers (Databricks) | London | up to £85k + Bonus Multiple roles Are you a Databricks expert ready to make a real impact? Join our high-growth consultancy and help shape the future of data platforms for some of the UK’s most asset-intensive industries. 📍 Location: London (Hybrid – 2 days/week onsite) 📄 Role Type: Permanent … base + up to 20% bonus (based on performance & commercial impact) 🔍 What’s the Opportunity? We’re on the lookout for a Mid/Senior Data Engineer to take a leading role in our expanding Databricks practice . With two senior team members moving on and demand growing fast, this is your chance to step into a high-impact … a values-driven consultancy with a strong presence in utilities, manufacturing, and aviation . Our team thrives on collaboration, technical excellence, and clear communication. We don’t just build data platforms—we solve real business problems. 🛠️ What You’ll Be Doing Designing and building scalable data pipelines using Databricks and medallion architecture Leveraging Delta Live Tables (DLT) for More ❯

to be partnering with a leading force in the UK Government & Defence space to help build a high-impact team delivering one of the most sensitive and strategically important data programmes in the country. Were seeking SC/DV Cleared Palantir Engineers who are driven by purpose, thrive in secure environments, and want to work on real-world challenges … ll Be Doing Youll be embedded in a fast-paced, mission-led programme where your work directly supports UK Government operations. Day-to-day, youll: Build and maintain scalable data pipelines within Palantir Foundry or Gotham Ingest, transform, and integrate both structured and unstructured sensitive data sources Enable secure access control, data modelling, and operational logic within … highly classified environments Collaborate with multidisciplinary teams of analysts, engineers, and delivery experts to design secure, data-driven solutions Contribute to a long-term agile delivery roadmap for mission-critical national programmes What We're Looking For We're interested in engineers who are motivated by impact , not just tech for tech's sake. You should have: Proven hands More ❯

to be partnering with a leading force in the UK Government & Defence space to help build a high-impact team delivering one of the most sensitive and strategically important data programmes in the country. We’re seeking SC/DV Cleared Palantir Engineers who are driven by purpose, thrive in secure environments, and want to work on real-world … Doing You’ll be embedded in a fast-paced, mission-led programme where your work directly supports UK Government operations. Day-to-day, you’ll: ⚙️ Build and maintain scalable data pipelines within Palantir Foundry or Gotham 🔄 Ingest, transform, and integrate both structured and unstructured sensitive data sources 🔐 Enable secure access control, data modelling, and operational logic within … highly classified environments 🤝 Collaborate with multidisciplinary teams of analysts, engineers, and delivery experts to design secure, data-driven solutions 🚀 Contribute to a long-term agile delivery roadmap for mission-critical national programmes 🧠 What We're Looking For We're interested in engineers who are motivated by impact , not just tech for tech's sake. You should have: ✅ Proven hands More ❯

options About the Role 🧠 A fast-growing UK tech group is looking for a hands-on AI Infrastructure Engineer to design and build the systems connecting core platforms (CRM, data lake, business tools) into a unified, AI-ready architecture. This is a greenfield build role — you’ll wire together SaaS tools, automate data flows, and shape the internal … AI Infrastructure/AI Platform Lead Hybrid London office: minimum 3 days onsite, hands-on role, fast-paced environment What You’ll Do 🔧 Design and implement end-to-end data pipelines linking core platforms (CRM → data lake → AI systems) Architect schemas and transformation layers optimized for analytics, automation, and AI workflows (RAG/internal copilots) Build Python and … to streamline operations and M&A integration processes Monitor, ensure reliability, and scale integrations and pipelines Document and modularize infrastructure for future AI engineers ✅ Must-Haves: Python & SQL experience Data engineering skills : pipeline design, data modeling, schema development Experience with API integrations, webhooks , and low/no-code automation tools (Zapier, Make, Matillion) Comfortable in AWS or More ❯

options About the Role 🧠 A fast-growing UK tech group is looking for a hands-on AI Infrastructure Engineer to design and build the systems connecting core platforms (CRM, data lake, business tools) into a unified, AI-ready architecture. This is a greenfield build role — you’ll wire together SaaS tools, automate data flows, and shape the internal … AI Infrastructure/AI Platform Lead Hybrid London office: minimum 3 days onsite, hands-on role, fast-paced environment What You’ll Do 🔧 Design and implement end-to-end data pipelines linking core platforms (CRM → data lake → AI systems) Architect schemas and transformation layers optimized for analytics, automation, and AI workflows (RAG/internal copilots) Build Python and … to streamline operations and M&A integration processes Monitor, ensure reliability, and scale integrations and pipelines Document and modularize infrastructure for future AI engineers ✅ Must-Haves: Python & SQL experience Data engineering skills : pipeline design, data modeling, schema development Experience with API integrations, webhooks , and low/no-code automation tools (Zapier, Make, Matillion) Comfortable in AWS or More ❯

Engineer Rate - £850 Hybrid working - London What you'll do Design and build AI prototypes using LLMs, agentic workflows, and vector-based knowledge retrieval Collaborate with investment teams and data scientists to identify and scope high-impact use cases Implement and orchestrate multi-agent systems using frameworks such as LangGraph or similar tools Develop pipelines and interfaces for prompt … engineering, context injection, and fine-tuned workflows Build internal demos and proof-of-concepts for investment research, market commentary, and portfolio analysis The knowledge, experience and qualifications you need Strong proficiency in Python, particularly for AI/ML, data engineering, or automation use cases Experience working with or building applications around large language models (open-source or … commercial) Familiarity with LLM agents, orchestration frameworks (e.g. LangGraph, LangChain), and context management Experience with cloud platforms (AWS preferred) and associated services for AI workloads A background in software engineering or data science, ideally in a research-driven or exploratory environment Strong collaboration skills and ability to work with investment stakeholders The knowledge, experience and qualifications that will More ❯

Engineer Rate - £850 Hybrid working - London What you'll do Design and build AI prototypes using LLMs, agentic workflows, and vector-based knowledge retrieval Collaborate with investment teams and data scientists to identify and scope high-impact use cases Implement and orchestrate multi-agent systems using frameworks such as LangGraph or similar tools Develop pipelines and interfaces for prompt … engineering, context injection, and fine-tuned workflows Build internal demos and proof-of-concepts for investment research, market commentary, and portfolio analysis The knowledge, experience and qualifications you need Strong proficiency in Python, particularly for AI/ML, data engineering, or automation use cases Experience working with or building applications around large language models (open-source or … commercial) Familiarity with LLM agents, orchestration frameworks (e.g. LangGraph, LangChain), and context management Experience with cloud platforms (AWS preferred) and associated services for AI workloads A background in software engineering or data science, ideally in a research-driven or exploratory environment Strong collaboration skills and ability to work with investment stakeholders The knowledge, experience and qualifications that will More ❯

Senior Software Engineer - Backend & Data | Fully Remote (UK) Founding Engineer | Python | GCP | Terraform | Event-Based Systems | Data Pipelines An innovative, mission-driven LegalTech SaaS start-up is looking for a Senior Product Engineer (Backend & Data) with deep experience in Python, Product development, and Data engineering. Role Information: Salary : Up to £100,000 (DOE) + Equity Location … only) Stack : Python , TypeScript, GCP, Pub/Sub, SQL & NoSQL, IaC (Terraform), CI/CD (GitHub Actions), Observability tools, AI tooling You’ll join a remote first, high-trust engineering team working with a modern, cloud-native stack - with real influence over technical decisions from day one. You’ll take technical ownership of the backend to ensure it’s … robust, scalable, and ready for real customers, while adding new Data-driven features, optimising performance, and shaping the long-term Product roadmap. While your focus will be backend systems in Python, you’ll also work across the stack, collaborate directly with users, and bring a strong Product mindset to every decision. This role is ideal for someone who thrives More ❯

SENIOR DATA ANALYST LOCATION HYBRID - LONDON (3 DAYS A WEEK IN OFFICE) SALARY UP TO 85,000 THE COMPANY A private equitybacked green tech business with major investment into data and analytics! THE ROLE As a Senior Analyst, you will be involved in the following: Delivering segmentation, regression, and predictive analysis to inform commercial decisions Building forecasting models … analysing customer and usage patterns Supporting dynamic pricing initiatives to optimise revenue and energy efficiency Translating analysis into clear, actionable insights for stakeholders across the business Working closely with data engineering and reporting teams to ensure high-quality data infrastructure YOUR SKILLS AND EXPERIENCE Strong skills in SQL and Python (AWS experience desirable) Background in statistical analysis More ❯

SENIOR DATA ANALYST LOCATION – HYBRID - LONDON (3 DAYS A WEEK IN OFFICE) SALARY – UP TO £85,000 THE COMPANY A private equity–backed green tech business with major investment into data and analytics! THE ROLE As a Senior Analyst, you will be involved in the following: Delivering segmentation, regression, and predictive analysis to inform commercial decisions Building forecasting … analysing customer and usage patterns Supporting dynamic pricing initiatives to optimise revenue and energy efficiency Translating analysis into clear, actionable insights for stakeholders across the business Working closely with data engineering and reporting teams to ensure high-quality data infrastructure YOUR SKILLS AND EXPERIENCE Strong skills in SQL and Python (AWS experience desirable) Background in statistical analysis More ❯

Senior Data Engineer (GenAI) - Leading Tech Company | London (Hybrid) Hybrid: 1 day in the office/4 days WFH Permanent role only Represented by Robert Half Limited Please note: No visa sponsorship on offer, No B2B,B2C, Fully remote workers or contractors on offer. The Opportunity We're looking for a Senior Data Engineer with hands-on GenAI … experience to join a forward-thinking technology business that's building data-driven solutions with real-world impact. This is a highly visible, senior-level role where you'll take ownership of data pipelines, architecture, and GenAI operational frameworks - helping shape the next generation of AI-driven products and analytics. If you enjoy combining engineering rigor with … AI innovation, this role gives you the platform, freedom, and backing to make it happen. Key Responsibilities Design, build, and maintain scalable data pipelines across modern cloud platforms Drive the implementation of GenAI frameworks, including model deployment, monitoring, and lifecycle management ('GenAI Ops') Develop robust ETL/ELT workflows and optimise performance within Snowflake Collaborate with cross-functional teams More ❯

Senior Software Engineer (Data-Centric) Manchester or South London (Hybrid – 2 x a week in office) £70,000 – £85,000 + 10% bonus + excellent benefits Our client is a global self-defence intelligence and technology organisation helping governments and defence agencies act with confidence. They use AI, graph databases, cloud, and search technologies to deliver faster, smarter decision … globally. Develop in one or more core technology areas: NoSQL (MongoDB, DocumentDB) Graph databases (GraphDB, Neo4j) Search technologies (ElasticSearch, OpenSearch, SOLR) ML/AI frameworks (SageMaker, Bedrock) Cloud-based data engineering (AWS) Build and maintain cloud-native data pipelines, ensuring data quality, accuracy, and reliability. What We’re Looking For 5+ years’ experience in software or … Proven ability to deliver reliable, scalable systems using cloud technologies (AWS preferred). Strong understanding of system design , API-first development , and SDLC best practices . Experience with complex data systems - structured, unstructured, or graph-based. Curious, proactive, and comfortable “rolling your sleeves up” to solve hard problems. Why Join? Work on advanced data and intelligence platforms that More ❯

Senior Software Engineer (Data-Centric) Manchester or South London (Hybrid – 2 x a week in office) £70,000 – £85,000 + 10% bonus + excellent benefits Our client is a global self-defence intelligence and technology organisation helping governments and defence agencies act with confidence. They use AI, graph databases, cloud, and search technologies to deliver faster, smarter decision … globally. Develop in one or more core technology areas: NoSQL (MongoDB, DocumentDB) Graph databases (GraphDB, Neo4j) Search technologies (ElasticSearch, OpenSearch, SOLR) ML/AI frameworks (SageMaker, Bedrock) Cloud-based data engineering (AWS) Build and maintain cloud-native data pipelines, ensuring data quality, accuracy, and reliability. What We’re Looking For 5+ years’ experience in software or … Proven ability to deliver reliable, scalable systems using cloud technologies (AWS preferred). Strong understanding of system design , API-first development , and SDLC best practices . Experience with complex data systems - structured, unstructured, or graph-based. Curious, proactive, and comfortable “rolling your sleeves up” to solve hard problems. Why Join? Work on advanced data and intelligence platforms that More ❯

Principal Python Engineer Salary: Up to £130k + bonus Location: London (3x per week) I’m working with a global financial organisation that’s investing heavily in its data and cloud technology platform, and they’re looking for a Python Engineer to join their growing Investment Technology team. You’ll be: Building clean, scalable Python back-end systems Designing … cloud-first solutions in AWS (Lambda, AppSync, DynamoDB) Collaborating with data, investment, and engineering teams Shaping best practices across CI/CD, DevOps, and agile development What we’re looking for: 5+ years’ experience in software engineering (3+ in Python) Needs to have experience within financial services, fintech, or investment tech! Strong AWS and data engineering … of APIs, SQL/NoSQL, and modern software design A problem-solver who enjoys working closely with the business If you’re a Python engineer who loves solving complex data challenges in a collaborative, cloud-first environment — drop me a message: daniel.wexler@sourcegroupinternational.com THIS ROLE DOES NOT PROVIDE SPONSORSHIP More ❯