1 to 25 of 63 Permanent Data Ingestion Jobs in London

Generation (RAG) layer — not a simple "search and stuff" pipeline, but an intelligent, self-directing retrieval system that can plan queries, route across heterogeneous data sources, assess and refine its own results, and feed high-fidelity context to downstream agents. The data landscape is complex: unstructured research, regulatory … documents, structured trade and risk data in relational and tabular stores, graph-based entity relationships, real-time market data, and internal knowledge bases. The RAG layer must unify all of this behind a coherent, agentic retrieval interface. Role Purpose They are hiring an engineer-architect who will ...

Data Bricks Product Owner, Microsoft Fabric Product Owner, Data Engineering, AI Data Analytics, Microsoft Stack, Azure, Data Bricks, ML, Azure, Mainly Remote Data Bricks Product Owner, Microsoft Fabric Project Owner, Data Engineering/Technology Product Owner required to join a global Professional Services business … remote role, but when travel is required (to London, Europe and the States) on occasions. We need someone who has come from a Data Bricks, Microsoft Fabric, Data-Engineering First background with a hardened skillset in Microsoft Stack Technologies (C# .NET Core, Fabric) who has then transitioned into ...

Senior Data Engineer Location : London (Hybrid) Type : Permanent Salary : £110,000 - £140,000 A global energy and commodities trading organisation is looking for a Senior Data Engineer to join its growing Data, AI & Analytics team in London. This is an opportunity to work at the heart … highly data-driven trading environment, building the platforms and pipelines that power advanced analytics, machine learning and systematic trading strategies. The firm operates across global energy and commodities markets and is known for combining quantitative research, advanced analytics and modern technology to support trading and risk management. Data ...

Data Architect Charity/Not for Profit Sector | £65,000£75,000 (12 mth FTC) | London (Hybrid) Our client is a charitable organisation with a clear mission and a growing ambition to put data at the heart of how it operates. Theyre looking for a hands-on Data Architect to take ownership of their data function at a pivotal moment in its development. This is a hands-on leadership position. Youll be responsible for the end-to-end data capability from governance frameworks and quality controls through to the architecture and delivery of our data ...

Data Architect – Charity/Not for Profit Sector | £65,000–£ mth FTC) | London (Hybrid) Our client is a charitable organisation with a clear mission and a growing ambition to put data at the heart of how it operates. They are looking for a hands-on Data Architect … take ownership of their data function at a pivotal moment in its development. This is a hands-on leadership position. You’ll be responsible for the end-to-end data capability — from governance frameworks and quality controls through to the architecture and delivery of our data warehouse ...

.No, we've not ventured into construction in our quest to conquer the world, rather a designer and builder of systems for all things data related where we are conquering the Data World. We can offer an interesting insight into projects spanning a variety of sectors, which … team where you can truly act as an expert in your field in support of our clients and their challenges in the world of data and technology, read on! The Team We're Intuita - a fast growing consultancy that's making waves in both the consultancy and technology space. ...

Senior Data Engineer Location: London (Hybrid) Function: Data Engineering/Data Platform Salary: Up to £140,000 base + bonus + benefits About the Role Our client is a global commodities and energy trading firm operating at the intersection of quantitative research, technology, and trading. Following … firm has significantly expanded its capabilities in renewable power optimisation, analytics, and physical gas trading across international markets. They are now hiring a Senior Data Engineer to join their Technology Data, AI & UX team in London. This team sits close to the trading floor and is responsible ...

Overview: We are working with a leading UK-based wealth and investment management firm seeking a skilled Data Engineer to support the build and evolution of their modern data platform. This role will focus on developing scalable data pipelines and contributing to a cloud-native lakehouse architecture. … Responsibilities: Design, build, and maintain robust data pipelines using Azure-based services Develop and optimize data workflows within Databricks Implement and manage data ingestion processes, including real-time streaming with Kafka Structure and maintain data across bronze, silver, and gold layers within a lakehouse architecture ...

Responsibilities Programme Delivery Lead delivery across multiple workstreams including platform, data ingestion, modelling, and reporting Manage an outsourced delivery partner across scope, timelines, budget, and quality Track KPIs, milestones, and governance forums (e.g. SteerCo, Design Authority) Proactively manage risks, issues, and interdependencies Data Architecture Define, implement … govern the target data architecture (platforms, data models, integrations) Ensure alignment with enterprise architecture and strategic objectives Review and challenge vendor designs to ensure quality and scalability Drive best practices across: Data modelling Data pipelines Storage (lakehouse/warehouse) Metadata, lineage, and data quality Operating ...

Role Summary Aubay is seeking a Principal Data Engineer to join a leading energy client’s team who will conceptualize and design data foundations and end-to-end solutions, including data platforms, to maximize value from data. The role promotes a data-driven culture across … teams and wider business stakeholders, acting as a subject matter expert and thought partner in defining the vision and data strategy for robust, future-ready IT solutions. Required skills and experience Strong proficiency with cloud platforms: AWS, Azure, SAP Expertise in ELT development and automation with scalable pipeline design ...

Contract Opportunity for an eDV Cleared Data Engineer with a leading National Security Consultancy in London. Active Enhanced DV Clearance required £680 p/d inside IR35 12-month engagement Full-time on-site in Central London The Role: You will design and deliver mission-critical data services … National Security clients. Working across AI, cyber security, cloud, big data and digital transformation initiatives, you will help shape the future of national security. You will build and manage robust data pipelines that convert raw, diverse data sources (batch, streaming, real-time and unstructured) into reliable, analysis ...

working with a leading global financial institution seeking a Lead Java engineer to help drive the development of a large scale, low latency market data platform. This role sits within a high impact data/tech function responsible for delivering real time and historical data to trading … quant teams. You will play a key role in shaping the architecture and evolution of a central data platform that underpins front office decision making. Key Responsibilities Lead the design and development of a large scale market data ingestion platform Build and optimise low latency data ...

Title: Data Engineer – Azure Data Platform Location : London (Hybrid) Our client is hiring a Data Engineer to help build and scale enterprise-grade data solutions on the Microsoft Azure platform. This role involves delivering reliable, high-performance data pipelines that support analytics and business operations. … Responsibilities: Design and develop ETL/ELT pipelines within Azure Build scalable solutions for data ingestion, transformation, and integration Ensure data quality, reliability, and performance Collaborate with cross-functional teams to deliver data-driven solutions Drive automation and optimization of data workflows Requirements: Experience ...

products which enables us to deliver, powerful, connected business solutions that drive significant client value. The Role: We are looking for a client-facing Data Consultant who can design, build, and optimise modern data platforms and analytics solutions using the Microsoft data stack, with a strong emphasis … Microsoft Fabric. You’ll work across data ingestion, transformation, modelling, governance, and visualisation—translating business needs into robust, secure, scalable solutions that deliver measurable outcomes. This person must be comfortable speaking to client and internal teams on technical matters. The Consultant will be responsible for: Delivery & Engineering Design ...

Employment Type: Full-time About the Role We are seeking a highly experienced Databricks Architect to lead the design, development, and optimization of scalable data platforms and advanced analytics solutions. This role requires a deep understanding of Databricks Lakehouse architecture, big data engineering, cloud ecosystems, and enterprise data strategy. The ideal candidate will partner with business and technology leaders to deliver high-performance data systems that support analytics, AI/ML, and real-time insights. Key Responsibilities Architecture & Solution Design Architect end-to-end Databricks Lakehouse solutions including data ingestion, ETL/ELT pipelines ...

with a 2 core office days per week. 🌍 Office Location: Old Street, London, UK The Company At kleene.ai , we’re accelerating how businesses become data-driven. We believe every company—regardless of size—should be able to easily harness, analyze, and act on their data. Our platform simplifies data ingestion, modeling, and reporting so our customers can spend less time wrestling with pipelines and more time delivering insights. Once our customers have a strong data foundation in place, we help them go even further—deploying AI models that unlock advanced analytics, automate decision-making, and surface ...

Role: Data, Privacy & AI Security Consultant Location: London Level: Associate Manager, Consultant or Senior Analyst Salary: Competitive Salary & Package (Dependent on Experience) Please Note: Any offer of employment is subject to satisfactory BPSS and SC security clearance which requires 5 years continuous UK address history (typically including no periods … point of application. Note: The above information relates to a specific client requirement As a Team: We are a specialist Data Security and Quantum Security practice, helping organisations translate regulatory, policy and standards-based requirements into practical, scalable technical controls. Our work spans the full lifecycle, from interpreting industry ...

Data Architect - Databricks £120,000-£125,000 base + benefits London (Hybrid: 2-3 days per week onsite) We're looking for an experienced Data Architect who can shape and deliver large-scale analytics platforms using Databricks and modern Lakehouse principles. This role is ideal for someone … enjoys working directly with clients, influencing data strategy, and designing high-quality, scalable solutions that enable AI, advanced analytics, and enterprise reporting. You'll join a growing architecture team where you'll play a key part in defining best practices, leading engagements, and supporting clients at the early stages ...

Senior Data Engineer - Contract - London - 3 days on site - £520-550pd via Umbrella Contract until Feb 27 Key Responsibilities ETL Pipeline Architecture : Design, build, and maintain scalable ETL pipelines using Apache Airflow, Python, PySpark, and AWS Glue to transform complex financial datasets into actionable insights. Data Integration … Lead the integration of diverse data sources, ensuring data consistency, accuracy, and timely delivery to downstream systems. Containerization & Deployment: Utilize Docker for containerization and manage deployments within an AWS cloud environment (utilizing EKS or ECS). Technical Leadership: As a Principal Engineer, provide technical guidance, code reviews ...

Director of Data London, Hybrid (3–4 days per week onsite) Salary up to £140,000 This is a standout opportunity to shape and scale the data and AI vision for a global consumer engagement business! The Company They are a global marketing and customer engagement organisation known … designing experience‐led rewards programs that strengthen acquisition, loyalty, and brand affinity. The Role • Define the data strategy, translating commercial priorities into clear roadmaps and high‐value use cases. • Lead the design, delivery, and optimisation of data products across personalisation, predictive analytics, and generative AI. • Build and guide ...

client is seeking a Business Analyst to join their FinOps product team, a newly created role sitting at the intersection of finance, operations and data engineering. Responsibilities: Gather and translate international business requirements into clear technical specifications and user stories for the FinOps product team Write and maintain … queries to extract, analyse and validate data across financial and operational databases Support data pipelines including bordereaux ingestion, data transformation and downstream reporting Provide hands-on technical product support, triaging issues and investigating data discrepancies Define acceptance criteria, lead UAT and validate delivered functionality against ...

Java Software Engineer – Data Engineering Up to £100,000 + bonus and benefits 4:1 hybrid in office – City of London Quant Capital is hiring for a small, high-performing market maker in London. This is a junior-to-mid role focused primarily on Java development, with a secondary … responsibility owning straightforward Python ETL/data ingestion into AWS and PostgreSQL. This is a growing team focused on data ingestion and building tooling for the wider firm. What you’ll do: Build and enhance Java backend services (APIs, batch jobs, integrations) used across the business ...

long term solo individual contributor role, within 12–18 months, you will grow into leading a team of 4–6 engineers across backend, data architecture, DevOps, and product engineering. What You'll Do Engineering & Architecture Design and build scalable, secure backend services and APIs for clinical AI inference … data processing Own cloud infrastructure (AWS), deployment pipelines, monitoring, and security controls Collaborate with the AI research team to deploy and serve machine learning models in production Establish engineering best practices across code review, testing, CI/CD, and documentation Data Architecture & Integration Design and maintain the data ...

Points Lead Data Engineer position Property Managing AI Solution – Start Up For a Technical Leader who enjoys building products with a passion for AI innovation Career path into a CTO role About The Company Seed funded AI solution for Property Management sector End to end automation removing all administrative … tasks for commercial and residential property businesses London HQ About the Role Architect and build scalable data pipelines and infrastructure to support AI and product systems. Design and maintain data ingestion, transformation, and storage architectures for operational and AI workloads. Develop and manage batch and real-time ...

London/City £70-100k salary + equity package Our client a global technology leader is currently looking for a Senior/Lead Data Engineer to work with the dev team to guide the provision of Software Development for an exciting new AI product. Key Responsibilities:- Architect … build scalable data pipelines and infrastructure Design and maintain data ingestion, transformation, and storage architectures for operational and AI workloads. Develop and manage batch and Real Time data pipelines. Build and optimize systems for vector search, retrieval, and ML data pipelines. Ensure data reliability ...