251 to 275 of 288 Data Pipeline Jobs in London

threats. As a Senior Backend Engineer , youll play a key role in building scalable, high-performance backend systems that ingest, process, and serve data on millions of assetshelping global security teams uncover exposures before attackers do. What youll be doing: Designing and building backend services and data pipelines powering a cutting-edge cybersecurity platform Architecting scalable microservices and APIs to process massive volumes of security data Working closely with product and frontend teams to integrate backend services into the platform experience Continuously improving systems performance, deployment, and monitoring as the company scales globally ...

industry-leading, AI-first technology platform in the intellectual property space is hiring a Staff-Level ML Engineer to lead all Data and NLP initiatives. This company was an early adopter of Large Language Models and is building advanced, agentic AI workflows that autonomously answer complex research questions. … models with cutting-edge NLP techniques to deliver high-precision search and insights at scale. The Role As the technical lead for Data & NLP, you will: Own and improve search performance (precision & recall) Optimise large-scale search algorithms for speed and relevance Design and implement agentic LLM workflows ...

Develop or apply cutting-edge techniques across: LLM prompt engineering & optimisation Graph-based and personalised information retrieval Contextual search and recommendation systems Synthetic data pipelines & LLM evaluation frameworks Build scalable methods and prototypes that can transition into real product environments. Analyse and document methodologies with the potential … publication in respected AI research venues. Support dataset creation, feature exploration and data preparation workflows. Deep-dive into your research area and share knowledge with the immediate team. Required Qualifications PhD/Doctorate or Currently Pursuing in Computer Science, Machine Learning, Artificial Intelligence, or a related field. Strong ...

pipelines for large, complex datasets • Infrastructure for distributed training and high-performance compute workloads • Model deployment systems, including inference services and batch pipelines • Data pipelines for ingestion, transformation, and versioning of large datasets • Monitoring, observability, and evaluation systems for ML performance in production This is a deeply technical … strong Python skills and experience building backend or distributed systems • Has worked with ML frameworks such as PyTorch, TensorFlow, or JAX • Understands training pipelines, data workflows, and model deployment • Has experience with cloud infrastructure (AWS/GCP) and containerization (Docker/Kubernetes) • Thinks in terms of systems, scalability ...

thrive in a high-performance startup environment. Key Responsibilities Design and build full-stack integrations connecting AI agents to customer systems Develop data pipelines and “glue code” for production deployments Run technical architecture sessions with customer stakeholders Act as the product feedback loop — surfacing bugs, bottlenecks, and feature … opportunities Manage technical relationships with IT, security, and data teams Write clean, maintainable, well-documented code Support continuous platform improvement and occasional core platform work What We’re Looking For 3+ years’ experience in Forward Deployed, Solutions, or Full-Stack Engineering Experience within a highly regulated Industry such ...

equivalent tooling. Back-end experience (e.g. Node.js or similar) with RESTful or GraphQL APIs. Experience integrating or working with Salesforce (APIs, webhooks, or data pipelines). Good understanding of application architecture, performance, and security. Comfort working in a small, fast-moving startup, taking ownership from idea to deployment. … internal assistants, or workflow automations). Experience with CI/CD, cloud platforms, and DevOps practices. Knowledge of SQL/NoSQL databases and data modelling. Previous work on tools for trades, field services, logistics, or other operations-heavy environments. What you'll get Real ownership over core product ...

Hedge Fund client is looking for a Python Developer/Engineer Contract role This team is responsible for the firms equity transaction data platform, including trade life cycle event processing, enrichment, and PnL calculations. The role is ideal for an engineer who enjoys building robust, high-throughput services … data pipelines in a fast-paced, delivery-focused environment. Principal Responsibilities Design and develop solutions for trade life cycle event processing, including corporate actions, expiries, and other post-trade events. Build and operate Python-based services that perform large-scale data transformations and calculations. Publish ...

Opportunity You’ll work across the stack (backend-leaning), helping build and evolve a platform that ingests and processes large-scale intelligence data used by security, finance and resilience teams. You’ll: Build and maintain backend services (TypeScript/Node.js) Develop and extend APIs Contribute to high-volume … data ingestion and processing workflows Ship frontend features in React/TypeScript when needed Write clean, well-tested, well-documented code Participate actively in code reviews and engineering standards Collaborate closely with Product and Engineering leadership What They’re Looking For 3+ years backend experience (strong Node.js/ ...

Technical Product Manager (Indices & Market Data) £110,000 - £125,000 + package London - Hybrid working model My leading FinTech client are looking for a talented and motivated individual with gravitas to own technical product areas across the platform (data pipelines, index calculation engines, internal tooling … complex data/platform products Strong technical background; comfortable discussing and reviewing: APIs and service boundaries, Data schemas and contracts, Pipeline architecture and failure modes, System design trade-offs. Domain experience in indices, capital markets, equities, derivatives, or other quantitative products with a track record ...

Role We are looking for a Head of Engineering to join a high-growth technology business operating at the forefront of large-scale data processing, machine learning and next-generation search experiences. In this role, you will lead multiple engineering teams across data … diverse and deeply technical organisation. Responsibilities Own the execution of the engineering roadmap across all product and platform areas Lead architectural direction for data processing, machine learning pipelines and web application development Ensure systems are scalable, reliable and capable of handling large-scale data and evolving ...

Python Engineer Sports Trading What you’ll tackle Translate data-flow headaches into buildable tech projects, managing data pipelines Python engineering to stand-up analytics services, REST APIs and lightweight front-ends. Data ingestion (SQL, files, feeds), business logic, web/UI, CI pipelines … with quants, ask clarifying questions, iterate live and keep comms flowing. What you need 1 + years building production-grade Python systems for data-heavy businesses; comfortable with tests, version control and optimisation. Confident SQL , exposure to REST and messaging (Rabbit, Kafka, etc.), and enough Linux to troubleshoot ...

multi-award-winning research agency in mid-transformation. Were rebuilding our processes from the inside out, including using Displayr to streamline our data pipeline. The Role Weve moved to Displayr, and need someone who can use it with confidence, curiosity and creativity. You will be the connection between … teams: part Displayr specialist, part problem-solver, part data scientist. This is a role at the coal-face of our work. Youll manage an ever-shifting workload, keeping projects on track amid changing priorities and pipelines. Key Responsibilities Consistently delivering clean, accurate tables & reports Help us to maximise ...

multi-award-winning research agency in mid-transformation. We’re rebuilding our processes from the inside out, including using Displayr to streamline our data pipeline. The Role We’ve moved to Displayr, and need someone who can use it with confidence, curiosity and creativity. You will … connection between teams: part Displayr specialist, part problem-solver, part data scientist. This is a role at the coal-face of our work. You’ll manage an ever-shifting workload, keeping projects on track amid changing priorities and pipelines. Key Responsibilities Consistently delivering clean, accurate tables & reports Help ...

standards, and regulatory requirements. Key Responsibilities: Enterprise AI Architecture & Roadmap: Define and maintain the AI reference architecture and implementation roadmap, covering LLM platforms, data pipelines, knowledge graphs, APIs, microservices, and event-driven architectures across hybrid cloud environments. Governance & Standards: Establish AI governance frameworks, policies, and playbooks for model … development, data usage, security, ethical AI, and compliance. Strategic Leadership: Provide architectural leadership and direction across delivery teams, ensuring consistent adoption of AI standards and patterns. Platform Integration: Guide the integration of AI capabilities into core enterprise platforms and processes, enabling reusable AI services and accelerators. Security & Risk ...

Senior/Lead Go Engineer – 📍 London | Full-Time | Fintech/Crypto I'm partnered with a leading global media and data platform in the digital asset space, who are looking for a Senior or Lead Go Engineer to join their London-based engineering team. This is a hands … opportunity to work with high-throughput systems delivering real-time market data that powers the next generation of finance and investing. You’ll play a key role in designing, building and optimising systems that aggregate, process and deliver real-time cryptocurrency and financial market data. The position will ...

About the Company A fast-growing fintech company based in London is hiring a Senior Python Engineer to work on real-time data systems and external data integrations. Engineers work with full autonomy across the entire development lifecycle in a fast-moving environment where technical decisions … have direct commercial impact. About the Role What You'll Work On Reverse-engineering third-party data sources and building robust integrations, including circumventing bot-protection mechanisms. Developing and maintaining low-latency, real-time data pipelines that underpin business-critical operations. Improving system observability to surface ...

Engineer to support a high-profile secure programme delivering geospatial capability across complex operational environments. Key responsibilities Design, build and maintain GIS data pipelines and spatial datasets Work across ArcGIS/ESRI tooling within secure cloud or on-prem environments Translate stakeholder operational requirements into geospatial technical solutions … Support integration with data platforms and analytical tooling Provide technical assurance and optimisation of spatial workflows Required experience Strong GIS engineering or geospatial data engineering background Experience working in SC cleared environments UK sole nationality required due to programme sensitivity Python, spatial ETL or data ...

Senior Data Engineer/GCP/Looker/Big Query £500 - 550 per day (Outside IR35) London/Hybrid Senior Data Engineer/GCP/Looker/Big Query This is an opportunity for you to join a rapidly growing ecommerce business who are looking … Senior Data Engineer to build and maintain data pipelines for a new reporting solution within the business. You'll be responsible for the build across ingestion, modelling and creating semantic layers using their GCP stack. Successful applicants will have experience with: Looker BigQuery GCP Gemini Exceptional ...

Required Experience & Skills Core Stack: Proven expertise in building and scaling services using Java , Postgres , and BigTable on Google Cloud Platform (GCP) . Data Engineering: Hands-on experience managing data pipelines, specifically utilizing Scio and Flight . Domain Knowledge: Previous experience in E-commerce or Financial ...

deploying, and optimising Splunk Enterprise and Splunk IT Service Intelligence (ITSI) in complex hybrid Kubernetes/OpenShift environments. You will handle large-scale data onboarding, build advanced ITSI service models and monitoring views, tune platform performance, implement secure governance, and integrate with modern observability pipelines. The position supports … high-stakes enterprise setting. Key Responsibilities Design, deploy, and operate Splunk Enterprise and ITSI in hybrid Kubernetes/OpenShift environments. Onboard data at scale using HEC, Universal Forwarders/Deployment Server; align to Common Information Model (CIM); enforce RBAC, retention policies, and cost guardrails. Build ITSI service decompositions ...

Responsibilities/Objectives 1.Design, deploy, and operate Splunk Enterprise and ITSI within hybrid Kubernetes and OpenShift environments 2.Onboard and manage large-scale data ingestion using HEC, Universal Forwarders, and Deployment Serve 3.Align data models to CIM standards and implement RBAC, retention policies, and cost controls 4.Develop … trees, KPIs, thresholds, and visualisation tools 3.Deep understanding of Kubernetes and OpenShift observability including cluster metrics, logs, and event correlation 4.Experience with modern data pipelines and telemetry tools such as OpenTelemetry, Prometheus, Fluentd/Fluent Bit, Vector, and Kafka Desirable Skills/Experience 1.Candidates should be comfortable working ...

access and leverage property data. Our platform empowers property developers, land agents, architects, and planners with real-time insights, helping them make smarter, data-driven decisions when acquiring and developing land. In just five years, we’ve built one of the UK’s most advanced property data pipelines, integrating thousands of data sources and processing terabytes of information with less than 24-hour latency. Our cutting-edge platform is shaping the future of property intelligence. About The Role We’re looking for a Lead Full-Stack Engineer who will combine deep technical ability ...

further information about this position please apply. you will build the infrastructure that sits between our traders and the market - execution paths, data pipelines, and observability tooling that power trillions in annual notional volume. When a system performs at 3am under peak load, you will … ordering guarantees, exactly-once semantics, consumer group management Production observability: metrics (Prometheus/OpenTelemetry), distributed tracing, structured logging, and alert design CI/CD pipeline design including benchmarking gates, automated performance regression detection, and reproducible builds TDD as a design discipline - property-based testing, integration tests, chaos and fault ...

long way from systems that actually matter. ETrading, you will build the infrastructure that sits between our traders and the market - execution paths, data pipelines, and observability tooling that power trillions in annual notional volume. When a system performs at 3am under peak load, you will … ordering guarantees, exactly-once semantics, consumer group management Production observability: metrics (Prometheus/OpenTelemetry), distributed tracing, structured logging, and alert design CI/CD pipeline design including benchmarking gates, automated performance regression detection, and reproducible builds TDD as a design discipline - property-based testing, integration tests, chaos and fault ...

Dev. This is a forward deployed software engineering role with an embedded-with-customer edge. You’ll contribute to product development, integrate complex data, build custom tools, and work shoulder-to-shoulder with customers to enable them to accomplish their goals in the product. An FDE's success … with the platform and achieve their mission objectives. You’ll be responsible for everything from understanding core user needs to writing code for data integrations and custom applications. This position requires you to think like a startup cofounder or an early-stage CTO. You’ll dive into ambiguous ...