51 to 75 of 94 Data Pipeline Jobs in the City of London

small, high‐impact team building AI systems that power decision‐making across a global investment platform. You’ll work alongside experienced AI engineers, data scientists, and technologists to deliver real products used by investment teams and portfolio companies. This role is ideal for: PhD graduates in a STEM … computational experience; or Bachelor’s/Master’s graduates with 1–2 years of industry experience or relevant internships in machine learning, data engineering, or software engineering. You’ll contribute to designing, implementing, and deploying production-grade ML systems—ranging from NLP pipelines to model‐driven workflow automation. ...

modern engineering best practices. Build and maintain microservices-based systems , with a strong focus on scalability, resilience, and performance. Develop and optimise scalable data pipelines , supporting both batch and streaming workloads, using technologies such as Apache Spark . Work extensively with data technologies , leveraging Python … deliver high-quality analytical and data-driven solutions. Lead the design and delivery of data-centric applications , translating complex business and analytical requirements into well-architected technical solutions. Implement and integrate large language models (LLMs) , including: Utilising both proprietary and open-source models Fine-tuning models ...

technology to trade a broad range of energy, metals, and agricultural commodities globally. Our success is driven by cutting-edge analytics, high-frequency data processing, and a commitment to innovation. We are now looking for a Senior Python Engineer to join our dynamic team in London and drive … Python Engineer, you will be at the heart of our technology platform, building robust systems that enhance our trading strategies, risk management, and data analysis. You will collaborate closely with quants, data scientists, and traders, providing Python-based solutions that directly impact the profitability ...

across sportsbooks, lottery, casino, virtual gaming, and financial trading. Our vision is to shape the future of gaming by transforming operations into intelligent, data-driven solutions that deliver exceptional customer experiences and create sustainable value for all stakeholders. We believe in teamwork, knowledge sharing, and transparency with accountability. … Technical Stack Our data ecosystem is built for massive scale and diverse query patterns: - Primary Transactional: **PostgreSQL** (Self-managed on EC2/K8s and AWS RDS/Aurora). - Real-Time Analytics: **ClickHouse** (used for sub-second telemetry, high-throughput event logging, and real-time trading dashboards). ...

Role: Business Analyst – Retention Data (EMEA) Location: Hybrid – London (3 days office/2 days remote) ⏳ Contract Length: 12 months minimum Day Rate: £260/day (Inside IR35) I’m hiring for a Business Analyst – Retention Data to join a leading global tech company known … days per week in the London office, 2 days remote . ** MUST HAVE ** 1: Within your experience and clearly on your CV: Data Insight, what impact did you make. 2: We are looking for someone who can display on their CV that has analysed data they ...

Consulting Engineer London (Hybrid) SC Cleared £50 - 55,000 Overview: Anson McCade are working with a cutting‐edge data analytics consultancy that partners with major public and private sector organisations to solve complex, real‐world problems using advanced data and AI platforms. This is an opportunity … join a high‐impact engineering team operating at the intersection of data engineering, analytics, cloud platforms, and client delivery . You’ll work directly with customers, building and deploying solutions that drive meaningful outcomes. The Role As a Consulting Engineer, you’ll take ownership of solution delivery ...

Senior Customer Success Engineer, Adoption & Enablement London, Hybrid Up to £95,000 We’re partnering with a scaling AI and data platform helping organisations turn complex data into real, operational insight. The product is landing well. Now the focus is on embedding it into client environments … roadshows, and solution sessions to drive adoption • Translating client needs into clear requirements for technical teams • Embedding the platform into live workflows with data teams • Driving usage, value realisation, and expansion • Building scalable enablement and onboarding approaches Tech environment • APIs, data pipelines, and integration workflows, understanding ...

Have Experience with procedural workflows (Houdini, Substance, or similar DCC tools) Ability to create and optimize sprite sheets, textures, and mesh data pipelines Strong problem-solving mindset with a passion for innovation in real-time graphics What You’ll Get Work on visually stunning, globally recognized VR experiences ...

Lead Engineer/Technical Lead will own the end-to-end technical delivery of a data migration and platform modernisation programme in AWS. This role acts as the single technical point of accountability, setting architecture, defining standards, and coordinating delivery across workstreams. While the programme includes data … evolve the AWS/CDP architecture for data migration, ingestion, and downstream consumption Define and enforce engineering standards (Terraform, CI/CD, pipeline design, naming conventions) Lead platform engineering and infrastructure delivery (IaC, networking, security, environment setup) Oversee migration of ~8–10 enterprise databases into AWS using ...

Contract Duration: 6 Months Pay Rate: 250 to 270 GBP/Day via Umbrella(Inside IR 35) The Opportunity Are you a Senior Data Analyst who is tired of just "maintaining" dashboards? Our client is looking for a forward-thinking analyst to join a Global Customer Success Data … into core workflows to automate insights and enhance the customer journey across multiple global markets. If you are passionate about moving beyond manual data prep and into the world of AI-native analytics , this is the role for you. What You'll Do Build AI-Native Foundations: Work ...

Hiring: Data Engineer | Mission-Driven Fintech | Remote (UK based) Propel are proud to be partnering with a rapidly scaling global fintech that’s transforming financial access for immigrants and international communities across the world. As their lending business continues to grow, they're looking for a Data Engineer to help scale the data infrastructure that powers smarter, fairer credit decisions. If you're passionate about building robust data pipelines, enabling real-time underwriting, and turning complex datasets into production-ready insights — this is your chance to work at the heart ...

with 20+ years of experience , combining deep hands‐on engineering capability with strategic advisory skills. You will sit at the intersection of AI, data, product, and business , translating ambiguous commercial objectives into deployable, production‐grade solutions that directly support transaction strategy, value creation, and post‐merger integration. … cases Translate complex business and transaction challenges into concrete technical and architectural solutions Forward Deployment & Solution Delivery Lead forward deployment of AI, data, and analytics solutions directly within client environments Design and implement end‐to‐end solutions covering data ingestion, modeling, orchestration, and deployment Rapidly prototype ...

totaled $13.8 billion. Experience : Minimum 10+ Years Strong Knowledge in Hadoop, Kafka, SQL/NoSQL Specialization in designing and implementing large-scale data pipelines, ETL processes, and distributed systems Should be able to work independenty with minimal help/guidance Good Understanding of Airflow,Data Fusion … Data Flow Strong Background and experience in Data Ingestions,Transformation,Modeling and Performance tuning. One migration Experience from Cornerstone to GCP will be added advantage Suppport the design and development of BigData echosystem Experience in building complex SQL Queries Strong Communication Skills ...

systems (LangGraph or similar) Defining best practices across DevOps, CI/CD, and engineering standards Integrating Power Platform tools with enterprise systems and data sources Shaping technical roadmaps, architecture patterns, and reusable frameworks Driving observability, performance, and continuous improvement Collaborating with cross-functional teams including product … experience with GenAI/LLMs and machine learning Experience building AI agents/multi-agent systems Knowledge of APIs, event-driven systems, data pipelines, and security best practices Exposure to Power Platform (Power Apps, Power Automate, Power BI) is a plus Strong stakeholder management and ability to work ...

Your Responsibilities: Design and maintain scalable data pipelines using PySpark, Python, and distributed computing frameworks to support high‐volume data processing. Architect and optimize AWS-based data and AI infrastructure, ensuring secure, performant, and cost‐efficient ingestion, transformation, and storage. Develop, finetune, benchmark … prompt management, evaluation, and GenAI operations. Collaborate with cross-functional teams to integrate GenAI capabilities into production systems and ensure high-quality data, governance, and operational reliability Your Profile: Strong experience with PySpark, distributed data processing, and largescale ETL/ELT pipelines. Strong SQL expertise including ...

Database Data Engineer | London (3 days a week onsite) | up to £100,000 Peaple Talent have partnered with a forward-thinking financial services organisation transforming access to credit across the UK. With a strong reputation for customer trust and innovation, they’re now looking for a Data Engineer with deep database expertise to play a critical role in modernising and scaling their data platforms. 💻 The Role This is a database-heavy Data Engineer position where you’ll take ownership of designing, optimising and evolving production-critical systems across AWS. Particularly using ...

across architecture, infrastructure, integrations, and backend systems. ABOUT THE ROLE Your Impact Architect and implement core systems: cloud infrastructure, backend services, APIs, and data pipelines Build secure, scalable, and reliable components ready for real-world operational environments Lead integrations with external systems and support early data … tooling, documentation, and technical processes from the start Set up observability, monitoring, and performance systems Jump in wherever needed, from quick scripts and data cleaning to debugging production issues What You Bring 5+ years of engineering experience with strong architecture/complex backend systems experience Deep hands ...

pipelines, vector search solutions, and agent-based workflows Creating robust evaluation frameworks to measure accuracy, reliability, and performance Building scalable APIs, services, and data pipelines Implementing observability, monitoring, and feedback loops to continuously improve outputs Optimising models and prompts for latency, cost, and quality across different providers What … production systems Solid understanding of cloud environments (AWS, GCP, or Azure) Experience with distributed systems, CI/CD, testing, and deployment pipelines Strong data handling skills (SQL, pandas) and an understanding of evaluation strategies for AI systems Comfortable working in ambiguous environments and iterating on non-deterministic systems ...

week in our Waterloo office Experience level: At least five years in commercial software engineering 🚀 About Orchestra Orchestra is an AI-native data pipelining platform building the future of data orchestration and observability . We have built a declarative framework allowing data engineers … explicitly declare how their data pipelines behave. Think CI/CD but for data teams! Backed by VCs from San Francisco, New York and London with angels from across the data space and beyond. 👷 Engineering at Orchestra We care deeply about our users—their ...

Founding BE Data Engineer (Python, Postgres, LLMs) Salary up to £150k + Equity London Onsite Join a lean (under 10), early‐stage startup building sustainable, environmentally‐focused financial services. The focus of the solution is understanding where capital flows, and the data behind those decisions needs … accurate, traceable, and built with care. This is a true end‐to‐end data focused role. You'll own data wrangling from ingestion through to delivery, no hand‐offs, no downstream dumping. Full responsibility, real impact. The company is profitable and revenue focused, with clients that ...

Google Cloud, or Azure. Knowledge of containerization and orchestration tools like Docker and Kubernetes is a plus. Experience designing end to end ML data pipelines. Experience with database technologies such as PostgreSQL, MySQL, or MongoDB, Spark, Databricks and stream data processing such as Kafka, RedPanda … stay up-to-date with the state-of-the-art open source infrastructure solutions applicable to designing and improving large scale recommender systems, data engineering, and machine learning You have a Master’s or PhD in Computer Science, statistics, or an engineering field with 5+ years of experience ...

Insight Global’s client is hiring a Product Owner to lead the digital, CRM and data capabilities that will power our Away from Home (Away from Home) go‐to‐market strategy and execution model. You will be the single business owner accountable for designing, delivering and realizing commercial … value from digital assets, Salesforce CRM and external data (e.g. Circana). This role owns automation, integration and the business use of digital assets and data to drive targeting, lead qualification, customer management, performance tracking and commercial decision‐making. This is a hybrid model role ...

Python Developer (AI/Data Focus) – Financial Services We’re partnering with an innovative financial services firm with deep expertise across FX, Swaps, Options, and a range of complex financial products. Their technology team is building cutting-edge, data-intensive, event-driven systems that power real … time market data, analytics, and execution workflows — and they’re looking for a Full-Stack Python Engineer to join them. 💻 What you’ll be working on: • Designing, developing, and optimising full-stack applications • Building API-first, scalable backend services • Developing real-time data pipelines and event ...

partnering with a rapidly scaling Palantir implementation partner, expanding across all seniority levels, having secured a strong pipeline of projects across various industries. This is an exciting opportunity to specialise in one of the fastest-growing technology markets and collaborate with some of the most experienced leaders … field. Responsibilities: Develop and maintain data pipelines and ETL processes Analyse client challenges and design tailored solutions Collaborate on implementing AI and machine learning models Serve as a point of contact, providing consultative guidance and support Requirements: Proven programming skills in Python and JavaScript Familiarity with Palantir ...

organisation continues to scale its AI capabilities, they are looking for someone to lead and shape the technical direction of their data and NLP initiatives. The Role As a Staff Applied Scientist, you will act as the technical lead for data and NLP initiatives , driving improvements … recall) • Optimising search algorithms to deliver fast and accurate results • Designing and implementing agentic AI workflows powered by LLMs • Building and maintaining scalable data pipelines • Leading technical decision-making across ML and NLP projects Requirements • 8+ years of commercial engineering and/or data science experience ...