76 to 94 of 94 Data Pipeline Jobs in the City of London

Role You’ll take ownership of designing and delivering an end-to-end AI-driven reporting and automation platform, working closely with Data Science and operational stakeholders. This is a build-from-scratch opportunity — ideal for someone who enjoys solving real-world complexity and turning manual processes into … robust, automated systems. What You’ll Be Doing Designing and building an automated reporting platform from the ground up Creating intelligent workflows across data ingestion, validation, and reconciliation Implementing AI-driven and agent-based automation to streamline operations Introducing exception handling and human-in-the-loop systems where ...

Senior Technical Business Analyst to support an Analytics & Modelling Platform across Gas, LNG and Power markets. You’ll work with Quants, Data Scientists and Engineers to translate complex modelling and forecasting requirements into scalable technical solutions. What you’ll do Gather and define requirements for forecasting, scenario modelling … analytics Translate quantitative concepts into technical requirements and user stories Support design of data pipelines, models and analytics tools Work with Python-based analytics and complex datasets Drive Agile delivery, UAT and stakeholder alignment What you’ll bring BA experience in energy trading, analytics or modelling Strong knowledge ...

Data Engineer 12 months Croydon - x1 day a week x4 remote Inside IR35 - Umbrella only Active SC clearance required Are you passionate about building reliable data pipelines and microservices? Our client, a leading organisation in the IT sector, is hiring for a talented Data ...

sophisticated Retrieval-Augmented Generation (RAG) systems, leveraging advanced techniques like embeddings with Voyage AI, rerankers , and query enrichment . Designing and maintaining efficient data pipelines and vector storage solutions using MongoDB Atlas Vector Search. Fine-tuning LLMs on custom datasets to enhance performance for specialized tasks related … Proven experience building and deploying applications with LLM APIs such as OpenAI , Anthropic , Gemini , and DeepSeek . Hands-on experience with the full RAG pipeline, including vector embeddings , rerankers , and data indexing in databases like MongoDB. Practical knowledge of LLM fine-tuning, prompt engineering, and performance optimization. ...

Senior Software Engineer | Sports Data London, UK | Remote Working Up to £80,000 + Company-wide benefits We’re partnered with a high-growth sports media network that operates a portfolio of market-leading brands and partnerships across the sports betting and tech space. Their mission … cutting-edge tools, all powered by modern, cloud-native technology. They are looking for an established Senior Software Engineer to join their expanding data team, where you will be working directly on key services that power a core part of the company's platform. While having essential responsibility ...

operational dashboards, and scripts that eliminate repetitive manual tasks across the business. Cross-Boundary Integration: Solve complex problems that span multiple systems, including data onboarding pipelines and API integrations. Product Delivery: Contribute directly to the core product, shipping high-quality features while ensuring the underlying systems remain robust. … modern CI/CD architectures, automated testing frameworks, and internal tool development. Application Code: Proficiency in building and maintaining production-grade product features. Data & Pipelines: Experience building ingestion workflows and connecting disparate internal data sources. Stack Versatility: You are highly proficient in Python ( FastAPI ) and TypeScript ...

team Requirements Published research in peer-reviewed venues such as IEEE, ICML, NeurIPS, ICLR, or equivalent Deep expertise in time series modelling - financial data experience strongly preferred Proven end-to-end ML experience: data pipelines, model development, deployment, and monitoring Strong Python and ML engineering skills … time and support for continued research and publishing Competitive base, performance bonus, and equity Hybrid working, flexible culture, and access to institutional-grade data and compute ...

most complex organisations in the world. We’re assembling a small, exceptional engineering team to solve problems at the intersection of AI, data, and scale—turning messy, manual workflows into automated, reliable systems that operate globally, securely, and at speed. If you’ve always wanted your work … core systems that make advanced AI workflows production‐ready: High‐performance backend services and APIs that power complex AI‐driven platforms Large‐scale data pipelines handling sensitive, high‐value datasets Systems that deploy and monitor machine‐learning models in real-world environments Enterprise‐grade infrastructure where reliability, security ...

arisen for an experienced KDB Developer to work alongside their Quants and Traders. This is an exciting opportunity to take ownership of critical data and analytics capabilities in a business where your work will directly support front-office decision-making, research, and platform performance. In addition … months. THE ROLE: KDB Developer This position sits within a small, highly capable team focused on building, improving and supporting the firms data environment. The successful candidate will play a key role in designing and developing scalable data collection and analytics solutions, while helping ensure ...

Role The Senior Technical Product Manager will own and build lifecycle and retention capabilities end-to-end, combining product thinking, technical execution, and data-driven experimentation to unlock sustained user growth. Role Highlights Design and launch AI-driven lifecycle journeys with minimal engineering support Architect and optimise … programmes across the full customer lifecycle Integrate APIs, tools, and data pipelines to deliver personalised experiences Run structured experiments to improve retention, engagement, and revenue Collaborate closely with growth, data, and engineering leadership You Will Need Proven ability to build and ship technical products independently Strong ...

tooling. Key responsibilities Design, deploy, and operate containerised services and internal tools used by trading desks and quants (Python‐based backend services, APIs, data pipelines). Maintain and optimise low‐latency systems in colocation environments, collaborating with trading and infrastructure teams on performance and resilience. Build and extend … with Prometheus ‐style observability: metrics, alerting, and dashboards; familiarity with Grafana is a plus. Hands‐on experience with ClickHouse or similar high‐performance data stores is a strong advantage. Practical experience with network automation ; Ansible or similar configuration‐management tools are beneficial. Experience running and maintaining container orchestration ...

powered product features, including the intent and routing logic that sits at the core of the platform – Own third-party integrations and data partnerships that expand our reach and signal quality – Keep the product knowledge base structured, current, and useful for the whole team – Contribute … products and made hard calls under ambiguity – Enough engineering ability to review code, prototype features, and pressure-test technical decisions – Comfortable with APIs, data pipelines, and integration work – Excellent written communication — specs, pitches, and user-facing copy – High EQ — able to work across engineers, designers, and external partners ...

real time. As the platform scales, they’re hiring a Full-Stack Engineer to design and deliver high-performance systems across real-time data pipelines, modern frontend applications, and AI-enhanced product features. What You'll Do as a Full-Stack Engineer: Design and build resilient, high-throughput … backend services in Java that power the movement of data across millions of sporting events each year. Architect and maintain real-time event-driven data pipelines and streaming systems at scale. Own cloud-native infrastructure decisions, working across serverless and containerised environments to deploy and operate ...

used by front-office teams Collaborate closely with traders and quantitative researchers to enable new trading strategies Build scalable, low-latency systems and data pipelines handling large volumes of market data Contribute across the full software development lifecycle, from design to production support Continuously improve system ...

Backend-Leaning | TypeScript/NodeJS/AWS) London (Paddington) | 3 days onsite £70k–£85k We’re working with a fast-growing, VC-backed data platform helping global brands understand and monetise user behavior in next-gen digital environments. The role Backend-leaning fullstack (80/20 split) Build … improve core data systems + APIs Work heavily with data pipelines (S3, transformations, high-volume data) Own features end-to-end (API/DB/internal tools) Help level up a growing engineering team Tech TypeScript, Node.js, React AWS (incl. S3), Kubernetes MySQL, MongoDB ...

understanding of .NET memory management internals - GC generations, LOH, pinned objects, finalisation, and how allocation patterns affect runtime behaviour Experience optimising high-throughput data processing pipelines - including async/await patterns, Channel, pipelines, and avoiding synchronisation bottlenecks Proven ability to diagnose large dataset query performance issues across both … allocation patterns, LOH fragmentation, finaliser queues, and GC tuning (gen0/1/2, server vs. Workstation, DATAS) Investigate and optimise high-throughput data pipelines - batch processing, streaming patterns, buffer management, and backpressure handling Identify and resolve large dataset query performance issues - index utilisation, query plan analysis, pagination ...

ALERT 🚨 – Lead Java Contractor 💻 Java/Distributed Systems/Event-Driven Architecture – Trading & Portfolio Tech 🕑 10+ years building high-performance, data-intensive trading systems 📍London (front-office, high-impact engineering environment) 💰 Highly competitive day rate Highlights: 🔷 Lead the design and build of scalable trading & portfolio management systems … real-time, business-critical technology 🔷 Architect event-driven, distributed systems with strong focus on performance, resilience, and observability 🔷 Drive technical direction across microservices, data streaming, and system design in a fast-moving environment The Role: Join a high-calibre Investment Engineering team embedded on the trading floor, delivering ...

Role We are looking for an experienced Principal Engineer to help scale a high-performance data processing and workflow platform used by enterprise customers. You will play a key, hands-on role working closely with technical leadership to: Evaluate and improve backend architecture for reliability, scalability, and security … abstractions to support rapid product and customer growth Scale infrastructure to handle increasing workloads and high-throughput processing Ensure robust handling of sensitive data in a secure, compliant manner Drive backend engineering best practices and build developer-friendly systems Contribute to long-term technical direction while remaining hands ...

discovery through sprint planning to build, ensuring architecture is integral and practical. Lead the AI design agenda by designing platforms, integration patterns, and data flows that support AI capabilities in the Specialty business. Interface with global Enterprise Architecture to adopt standards beneficial to the UK business while protecting … placing platforms and integration patterns (Whitespace, PPL, API ecosystem). Strong knowledge of market messaging standards such as ACORD (MRC, AL3) and structured data initiatives like Blueprint Two. Experience with post-bind and processing architecture including policy administration, endorsements, premium flows, and carrier settlement. Expertisein delegated authority system ...