476 to 500 of 876 Data Pipeline Jobs in London

Location: Central London (Hybrid options available) Daily Rate contract - Outside IR35 A leading FTSE 250 organisation is seeking a BI Fabric Developer/Engineer to help transition its data platforms into their Microsoft Fabric workspace. This is a unique opportunity to work on cutting-edge BI and analytics solutions that drive strategic decision-making across the business. Key … Responsibilities Lead the migration of data platforms into Microsoft Fabric for enhanced reporting and stakeholder access. Design and maintain data pipelines using Fabric Pipelines, Azure Data Factory, Notebooks and SSIS. Develop Power BI dashboards and paginated reports tailored to business needs. Model complex datasets into actionable insights using star schemas, snowflakes, and denormalised models. … teams to translate business requirements into technical solutions. Support CI/CD deployment processes using Azure DevOps. Required Skills & Experience Strong expertise in Power BI, DAX, SQL, and data architecture. Proven experience with Azure Data Factory, Fabric Lakehouse/Warehouse, and Synapse. Proficiency in PySpark, T-SQL, and data transformation using Notebooks. Familiarity with More ❯

extension) Start: ASAP Rate: Market rate - Inside IR35 We’re looking for experienced PySpark + Fabric Developers to join a major transformation programme with a leading global financial data and infrastructure organisation. This is an exciting opportunity to work on cutting-edge data engineering solutions, driving innovation and performance at scale. Key Responsibilities Design, build, and … optimise data pipelines for both batch and streaming workloads. Develop and manage dataflows and semantic models to support analytics and reporting. Implement complex data transformations, aggregations, and joins with a focus on performance and reliability. Apply robust data validation, cleansing, and profiling techniques to maintain accuracy. Enforce role-based access, data masking … hands-on experience with PySpark (RDDs, DataFrames, Spark SQL). Proven ability to build and optimise ETL pipelines and dataflows. Familiar with Microsoft Fabric or similar lakehouse/data platform environments. Experience with Git, CI/CD pipelines, and automated deployment. Knowledge of market data, transactional systems, or financial datasets. Excellent communication skills and collaborative mindset. More ❯

A leading Financial Services client in London is undergoing a major data transformation and is hiring a Data Engineering Manager on a permanent basis. The role offers a base of £85,000, strong benefits, and flexible hybrid/remote working. Were looking for a hands-on leader from a forward-thinking, modern data environment … with a strong foundation in Data Engineering and experience managing a squad of 47 Data Engineers. Youll bring solid SQL skills, cloud expertise (preferably Azure), and familiarity with tools such as Snowflake, Databricks, ADF, Python, and GenAI. Youll manage a squad of engineers and play a key role in delivering scalable, modern data solutions … as part of a group-wide transformation. Requirements: Proven experience in fast-paced environments working with modern data architectures and tooling; Financial Services experience is a plus. Experience managing Data Engineering teams Strong SQL and cloud skills (Azure preferred) Familiar with modern data tools (e.g. Snowflake, Databricks, ADF, Python) Track record of building/ More ❯

machine learning, deep learning or AI engineering • Advanced proficiency in Python and ML frameworks like PyTorch • Experience deploying and maintaining scalable production-grade AI systems • Strong understanding of data pipelines, MLOps and cloud platforms (AWS and Azure) • Experience working in small or mid-sized tech/software development environments (rather than in corporate environments) would be advantageous The More ❯

machine learning, deep learning or AI engineering • Advanced proficiency in Python and ML frameworks like PyTorch • Experience deploying and maintaining scalable production-grade AI systems • Strong understanding of data pipelines, MLOps and cloud platforms (AWS and Azure) • Experience working in small or mid-sized tech/software development environments (rather than in corporate environments) would be advantageous The More ❯

machine learning, deep learning or AI engineering • Advanced proficiency in Python and ML frameworks like PyTorch • Experience deploying and maintaining scalable production-grade AI systems • Strong understanding of data pipelines, MLOps and cloud platforms (AWS and Azure) • Experience working in small or mid-sized tech/software development environments (rather than in corporate environments) would be advantageous The More ❯

machine learning, deep learning or AI engineering • Advanced proficiency in Python and ML frameworks like PyTorch • Experience deploying and maintaining scalable production-grade AI systems • Strong understanding of data pipelines, MLOps and cloud platforms (AWS and Azure) • Experience working in small or mid-sized tech/software development environments (rather than in corporate environments) would be advantageous The More ❯

Build and maintain Python-based tools supporting mortgage analytics, valuation, and reporting. • Develop production-grade applications and contribute to the migration of legacy models. • Identify automation opportunities across data pipelines, analytics, and reporting. • Collaborate with business teams to gather requirements and deliver scalable solutions. • Champion best practices in testing, documentation, and maintainability. Requirements • Strong proficiency in Python • Experience More ❯

Build and maintain Python-based tools supporting mortgage analytics, valuation, and reporting. • Develop production-grade applications and contribute to the migration of legacy models. • Identify automation opportunities across data pipelines, analytics, and reporting. • Collaborate with business teams to gather requirements and deliver scalable solutions. • Champion best practices in testing, documentation, and maintainability. Requirements • Strong proficiency in Python • Experience More ❯

Build and maintain Python-based tools supporting mortgage analytics, valuation, and reporting. • Develop production-grade applications and contribute to the migration of legacy models. • Identify automation opportunities across data pipelines, analytics, and reporting. • Collaborate with business teams to gather requirements and deliver scalable solutions. • Champion best practices in testing, documentation, and maintainability. Requirements • Strong proficiency in Python • Experience More ❯

Build and maintain Python-based tools supporting mortgage analytics, valuation, and reporting. • Develop production-grade applications and contribute to the migration of legacy models. • Identify automation opportunities across data pipelines, analytics, and reporting. • Collaborate with business teams to gather requirements and deliver scalable solutions. • Champion best practices in testing, documentation, and maintainability. Requirements • Strong proficiency in Python • Experience More ❯

and optimise infrastructure to support high throughput training, inference and deployment of both multi agent reinforcement learning systems and enormous language models • Implement robust MLOps best practices including data pipelines, model versioning, monitoring, automated deployment and continuous integration • Drive end to end ownership from ideation to deployment, performance and scaling • Mentor future hires and define scalable engineering culture More ❯

and optimise infrastructure to support high throughput training, inference and deployment of both multi agent reinforcement learning systems and enormous language models • Implement robust MLOps best practices including data pipelines, model versioning, monitoring, automated deployment and continuous integration • Drive end to end ownership from ideation to deployment, performance and scaling • Mentor future hires and define scalable engineering culture More ❯

and optimise infrastructure to support high throughput training, inference and deployment of both multi agent reinforcement learning systems and enormous language models • Implement robust MLOps best practices including data pipelines, model versioning, monitoring, automated deployment and continuous integration • Drive end to end ownership from ideation to deployment, performance and scaling • Mentor future hires and define scalable engineering culture More ❯

and optimise infrastructure to support high throughput training, inference and deployment of both multi agent reinforcement learning systems and enormous language models • Implement robust MLOps best practices including data pipelines, model versioning, monitoring, automated deployment and continuous integration • Drive end to end ownership from ideation to deployment, performance and scaling • Mentor future hires and define scalable engineering culture More ❯

assistants - Summarization tools Apply prompt engineering, Retrieval-Augmented Generation (RAG), and context-aware pipelines to enhance model accuracy and relevance. Integrate AI models with enterprise systems, APIs, and data stores using Python, Java, or Node.js. Collaborate with architects to define scalable, secure, and cost-efficient AI service architectures. Implement AI/ML pipelines for training, validation, and deployment … model compression, quantization, and API optimization. Ensure compliance with AI ethics, security, and governance standards. Prepare and curate training datasets (structured/unstructured text, images, code). Apply data preprocessing, tokenization, and embedding generation techniques. Work with vector databases (e.g., Pinecone, Weaviate, FAISS, Chroma) for semantic search and retrieval. Partner with business stakeholders to identify and shape impactful … with cloud platforms (AWS, GCP, Azure) and AI/ML services. Knowledge of MLOps tools and practices (e.g., MLflow, Kubeflow, Vertex AI, Azure ML). Strong understanding of data engineering, data pipelines, and ETL workflows. Excellent problem-solving, communication, and stakeholder engagement skills. Bachelor's or Master's degree in Computer Science, AI/ML, Data More ❯

assistants - Summarization tools Apply prompt engineering, Retrieval-Augmented Generation (RAG), and context-aware pipelines to enhance model accuracy and relevance. Integrate AI models with enterprise systems, APIs, and data stores using Python, Java, or Node.js. Collaborate with architects to define scalable, secure, and cost-efficient AI service architectures. Implement AI/ML pipelines for training, validation, and deployment … model compression, quantization, and API optimization. Ensure compliance with AI ethics, security, and governance standards. Prepare and curate training datasets (structured/unstructured text, images, code). Apply data preprocessing, tokenization, and embedding generation techniques. Work with vector databases (e.g., Pinecone, Weaviate, FAISS, Chroma) for semantic search and retrieval. Partner with business stakeholders to identify and shape impactful … with cloud platforms (AWS, GCP, Azure) and AI/ML services. Knowledge of MLOps tools and practices (e.g., MLflow, Kubeflow, Vertex AI, Azure ML). Strong understanding of data engineering, data pipelines, and ETL workflows. Excellent problem-solving, communication, and stakeholder engagement skills. Bachelor's or Master's degree in Computer Science, AI/ML, Data More ❯

assistants - Summarization tools Apply prompt engineering, Retrieval-Augmented Generation (RAG), and context-aware pipelines to enhance model accuracy and relevance. Integrate AI models with enterprise systems, APIs, and data stores using Python, Java, or Node.js. Collaborate with architects to define scalable, secure, and cost-efficient AI service architectures. Implement AI/ML pipelines for training, validation, and deployment … model compression, quantization, and API optimization. Ensure compliance with AI ethics, security, and governance standards. Prepare and curate training datasets (structured/unstructured text, images, code). Apply data preprocessing, tokenization, and embedding generation techniques. Work with vector databases (e.g., Pinecone, Weaviate, FAISS, Chroma) for semantic search and retrieval. Partner with business stakeholders to identify and shape impactful … with cloud platforms (AWS, GCP, Azure) and AI/ML services. Knowledge of MLOps tools and practices (e.g., MLflow, Kubeflow, Vertex AI, Azure ML). Strong understanding of data engineering, data pipelines, and ETL workflows. Excellent problem-solving, communication, and stakeholder engagement skills. Bachelor's or Master's degree in Computer Science, AI/ML, Data More ❯

Senior Data Engineer (AWS Kafka Python) London/WFH to £85k Are you a tech savvy Data Engineer with AWS expertise combined with client facing skills? You could be joining a global technology consultancy with a range of banking, financial services and insurance clients in a senior, hands-on Data Engineer role. As a … Senior Data Engineer you will design and build end-to-end real-time data pipelines using AWS native tools, Kafka and modern data architectures, applying AWS Well-Architected Principles to ensure scalability, security and resilience. You’ll collaborate directly with clients to analyse requirements, define solutions and deliver production grade systems, leading the development … of robust, well tested and fault tolerant data engineering solutions. Location/WFH: There’s a hybrid work from home model with two days a week in the London, City office (or at client site in London). About you: You are an experienced Data Engineer within financial services environments You have expertise with AWS including More ❯

Senior Data Engineer (AWS Kafka Python) London/WFH to £85k Are you a tech savvy Data Engineer with AWS expertise combined with client facing skills? You could be joining a global technology consultancy with a range of banking, financial services and insurance clients in a senior, hands-on Data Engineer role. As a … Senior Data Engineer you will design and build end-to-end real-time data pipelines using AWS native tools, Kafka and modern data architectures, applying AWS Well-Architected Principles to ensure scalability, security and resilience. You'll collaborate directly with clients to analyse requirements, define solutions and deliver production grade systems, leading the development … of robust, well tested and fault tolerant data engineering solutions. Location/WFH: There's a hybrid work from home model with two days a week in the London, City office (or at client site in London). About you: You are an experienced Data Engineer within financial services environments You have expertise with AWS including More ❯

We’re seeking a Lead Data Engineer to design and deliver scalable data and AI solutions across internal and client projects. You’ll own the full data lifecycle — from ingestion and processing to deployment and monitoring of machine learning models — while leading a small, high-performing team. What You’ll Do Build and maintain … data pipelines and cloud infrastructure (AWS/GCP). Develop AI-driven applications including LLM tools, RAG pipelines, and automation solutions. Lead on client communication , project delivery, and stakeholder alignment. Collaborate with Data Science and Analytics teams to improve data access, automation, and governance. Drive innovation through proofs-of-concept , model monitoring , and adoption … of emerging AI tech. What We’re Looking For 5+ years in data engineering, with team or project leadership experience. Advanced Python (Pandas, PyTorch, TensorFlow, Scikit-learn). Strong AWS/GCP , MySQL , and CI/CD experience; Docker/Kubernetes a plus. Excellent communication, organisation, and problem-solving skills. Passion for innovation and ethical, scalable AI. Nice More ❯

Job Description: Locality Expert Employment Type: Full-time Work Type: Remote Duration: 6 months Location: London Role Objective: The team works on keeping Maps Data current and reflective of real-world changes. They work on issues submitted by end users of Maps or test the location results of user queries submitted via Web Search or Maps interface. The … Mapping Maintenance Expert will be responsible for improving & maintenance of city/country transit data and providing country specific expertise. The Maintenance Expert will be responsible for developing and maintaining transit data pipelines and establishing the ground truth for the countries they would be responsible for maintenance. Role & Responsibilities as a Maintenance Expert: ❏ Communicating with City … Experts & other stakeholder teams and achieving the project objectives. ❏ Identify transit user expectations, broken critical user journeys, data gaps and provide insights to the product. ❏ Validate quality of new and updated data in the data management platform. ❏ Make necessary edits for all transit data types. ❏ Flag issues to Program Managers to communicate More ❯

Job Description: Locality Expert Employment Type: Full-time Work Type: Remote Duration: 6 months Location: London Role Objective: The team works on keeping Maps Data current and reflective of real-world changes. They work on issues submitted by end users of Maps or test the location results of user queries submitted via Web Search or Maps interface. The … Mapping Maintenance Expert will be responsible for improving & maintenance of city/country transit data and providing country specific expertise. The Maintenance Expert will be responsible for developing and maintaining transit data pipelines and establishing the ground truth for the countries they would be responsible for maintenance. Role & Responsibilities as a Maintenance Expert: ❏ Communicating with City … Experts & other stakeholder teams and achieving the project objectives. ❏ Identify transit user expectations, broken critical user journeys, data gaps and provide insights to the product. ❏ Validate quality of new and updated data in the data management platform. ❏ Make necessary edits for all transit data types. ❏ Flag issues to Program Managers to communicate More ❯

We're seeking a Lead Data Engineer to design and deliver scalable data and AI solutions across internal and client projects. You'll own the full data life cycle - from ingestion and processing to deployment and monitoring of machine learning models - while leading a small, high-performing team. What You'll Do Build and … maintain data pipelines and cloud infrastructure (AWS/GCP). Develop AI-driven applications including LLM tools, RAG pipelines, and automation solutions. Lead on client communication , project delivery, and stakeholder alignment. Collaborate with Data Science and Analytics teams to improve data access, automation, and governance. Drive innovation through proofs-of-concept , model monitoring , and … adoption of emerging AI tech. What We're Looking For 5+ years in data engineering, with team or project leadership experience. Advanced Python (Pandas, PyTorch, TensorFlow, Scikit-learn). Strong AWS/GCP , MySQL , and CI/CD experience; Docker/Kubernetes a plus. Excellent communication, organisation, and problem-solving skills. Passion for innovation and ethical, scalable AI. More ❯

when needed. The ideal candidate will act primarily as a Solutions Architect - collaborating with clients, consultancy teams, and internal stakeholders to design scalable, future-proof solutions across digital, data, and MarTech domains. However, this individual should also be comfortable getting hands-on in engineering scenarios, whether it's during critical build phases, technical troubleshooting, or prototyping new concepts. … technical ideas into business value. Solution Design & Delivery Translate business requirements into architectural designs and actionable technical documentation. Design robust, scalable, and maintainable digital solutions across web, mobile, data, and MarTech ecosystems. Take ownership of high-level designs, low-level designs, integration specifications, and build oversight. Client Collaboration & Delivery Execution Participate in workshops and discovery sessions to capture … and non-technical audiences. Nice to Have Experience with platforms like Braze, Google Analytics, Adobe Analytics, Contentful, or Sitecore. Experience across technology domains including CMS, CRM, Martech platforms, data pipelines, analytics, and cloud services. Exposure to CDPs (e.g., Bloomreach, Segment, BlueConic, Tealium) and data integration pipelines. Understanding of data modelling, ETL processes, and basic More ❯