651 to 675 of 980 Data Pipeline Jobs in London

be partnering with a leading force in the UK Government & Defence space to help build a high-impact team delivering one of the most sensitive and strategically important data programmes in the country. We re seeking SC/DV Cleared Palantir Engineers who are driven by purpose, thrive in secure environments, and want to work on real-world … You ll be embedded in a fast-paced, mission-led programme where your work directly supports UK Government operations. Day-to-day, you ll: Build and maintain scalable data pipelines within Palantir Foundry or Gotham Ingest, transform, and integrate both structured and unstructured sensitive data sources Enable secure access control, data modelling, and operational … interested in engineers who are motivated by impact , not just tech for tech's sake. You should have: Proven hands-on experience in Palantir Foundry or Gotham deployment, configuration, pipeline development Strong background in data engineering , analytics , or data architecture Coding experience in Python, SQL , or similar Experience delivering in secure , high-stakes, or classified More ❯

Miryco Consultants are working on behalf of a market leading investment manager to hire a Data & AI Risk Analyst to join their second line Technology Risk team. This is a fantastic opportunity for a risk professional early in their career to specialise in a highly topical area, whilst gaining wider experience of working in a high-performance culture. … Responsibilities: Support the review and monitoring of risks related to data platforms, cloud environments, and emerging AI technologies. Assist in analysing technology incidents, change activities, and control testing to identify potential risk exposures. Contribute to the development of risk reporting, dashboards, and governance materials. Help assess how AI, automation, and data tools are being used across … the business from a risk perspective. Work with senior risk managers and engineers to promote strong data governance and AI control practices. Stay up to date with developments in AI, data protection, and tech regulation to help shape risk oversight. Experience & Skills: Early career professional or recent graduate with experience or academic background in data More ❯

Miryco Consultants are working on behalf of a market leading investment manager to hire a Data & AI Risk Analyst to join their second line Technology Risk team. This is a fantastic opportunity for a risk professional early in their career to specialise in a highly topical area, whilst gaining wider experience of working in a high-performance culture. … Responsibilities: Support the review and monitoring of risks related to data platforms, cloud environments, and emerging AI technologies. Assist in analysing technology incidents, change activities, and control testing to identify potential risk exposures. Contribute to the development of risk reporting, dashboards, and governance materials. Help assess how AI, automation, and data tools are being used across … the business from a risk perspective. Work with senior risk managers and engineers to promote strong data governance and AI control practices. Stay up to date with developments in AI, data protection, and tech regulation to help shape risk oversight. Experience & Skills: Early career professional or recent graduate with experience or academic background in data More ❯

Apply advanced analytical techniques, such as machine learning, natural language processing, computer vision, and deep learning, to extract insights and generate solutions from structured and unstructured data. Build data pipelines, models, and AI applications using cloud platforms and frameworks such as Azure AI/ML Studio, AWS Bedrock, GCP Vertex, Spark, TensorFlow, PyTorch, etc. Build and deploy production … grade fine-tuned LLMs and complex RAG architectures. Create and manage the complex and robust prompts across the GenAI solutions. Communicate effectively with stakeholders and colleagues, using data visualisation, storytelling, and presentation skills. Ensure the ethical use of AI and adherence to data privacy regulations. Essential Skills and Experience Experienced in data science, machine … learning and AI and delivering AI-driven solutions using a variety of tools and techniques Proficient in programming languages such as Python and familiar with data science, AI libraries and frameworks. Expert in implementing Agentic AI solutions , MCP protocols and integrating with GenAI based applications. Experience in working with cloud platforms and services, such as Azure, AWS and More ❯

Apply advanced analytical techniques, such as machine learning, natural language processing, computer vision, and deep learning, to extract insights and generate solutions from structured and unstructured data. Build data pipelines, models, and AI applications using cloud platforms and frameworks such as Azure AI/ML Studio, AWS Bedrock, GCP Vertex, Spark, TensorFlow, PyTorch, etc. Build and deploy production … grade fine-tuned LLMs and complex RAG architectures. Create and manage the complex and robust prompts across the GenAI solutions. Communicate effectively with stakeholders and colleagues, using data visualisation, storytelling, and presentation skills. Ensure the ethical use of AI and adherence to data privacy regulations. Essential Skills and Experience Experienced in data science, machine … learning and AI and delivering AI-driven solutions using a variety of tools and techniques Proficient in programming languages such as Python and familiar with data science, AI libraries and frameworks. Expert in implementing Agentic AI solutions , MCP protocols and integrating with GenAI based applications. Experience in working with cloud platforms and services, such as Azure, AWS and More ❯

Apply advanced analytical techniques, such as machine learning, natural language processing, computer vision, and deep learning, to extract insights and generate solutions from structured and unstructured data. Build data pipelines, models, and AI applications using cloud platforms and frameworks such as Azure AI/ML Studio, AWS Bedrock, GCP Vertex, Spark, TensorFlow, PyTorch, etc. Build and deploy production … grade fine-tuned LLMs and complex RAG architectures. Create and manage the complex and robust prompts across the GenAI solutions. Communicate effectively with stakeholders and colleagues, using data visualisation, storytelling, and presentation skills. Ensure the ethical use of AI and adherence to data privacy regulations. Essential Skills and Experience Experienced in data science, machine … learning and AI and delivering AI-driven solutions using a variety of tools and techniques Proficient in programming languages such as Python and familiar with data science, AI libraries and frameworks. Expert in implementing Agentic AI solutions , MCP protocols and integrating with GenAI based applications. Experience in working with cloud platforms and services, such as Azure, AWS and More ❯

Apply advanced analytical techniques, such as machine learning, natural language processing, computer vision, and deep learning, to extract insights and generate solutions from structured and unstructured data. Build data pipelines, models, and AI applications using cloud platforms and frameworks such as Azure AI/ML Studio, AWS Bedrock, GCP Vertex, Spark, TensorFlow, PyTorch, etc. Build and deploy production … grade fine-tuned LLMs and complex RAG architectures. Create and manage the complex and robust prompts across the GenAI solutions. Communicate effectively with stakeholders and colleagues, using data visualisation, storytelling, and presentation skills. Ensure the ethical use of AI and adherence to data privacy regulations. Essential Skills and Experience Experienced in data science, machine … learning and AI and delivering AI-driven solutions using a variety of tools and techniques Proficient in programming languages such as Python and familiar with data science, AI libraries and frameworks. Expert in implementing Agentic AI solutions , MCP protocols and integrating with GenAI based applications. Experience in working with cloud platforms and services, such as Azure, AWS and More ❯

with an organisation that’s passionate about your learning journey and committed to your professional career and personal development. Lead Technology Integration Workstream Drive the financial planning and data integration workstream within a broader transformation or modernization program. Align integration strategy with enterprise financial systems, ensuring data accuracy and consistency across planning, reporting, and forecasting platforms. … maintain a detailed roadmap covering milestones, dependencies, and risk mitigation plans. Oversee day-to-day execution, ensuring timely delivery and adherence to scope, quality, and compliance standards. Drive Data Integration, Migration, and Validation Lead the design and execution of data migration, validation, and reconciliation processes to support accurate financial and operational reporting. Collaborate with data and technology teams to ensure robust, secure data pipelines between core systems (e.g., ERP, EPM, data warehouse). Cross-Functional Coordination Serve as the bridge between Finance, Operations, Technology, and Data teams to ensure alignment on priorities, requirements, and timelines. Communicate effectively with stakeholders at all levels, translating technical concepts into actionable More ❯

with an organisation that’s passionate about your learning journey and committed to your professional career and personal development. Lead Technology Integration Workstream Drive the financial planning and data integration workstream within a broader transformation or modernization program. Align integration strategy with enterprise financial systems, ensuring data accuracy and consistency across planning, reporting, and forecasting platforms. … maintain a detailed roadmap covering milestones, dependencies, and risk mitigation plans. Oversee day-to-day execution, ensuring timely delivery and adherence to scope, quality, and compliance standards. Drive Data Integration, Migration, and Validation Lead the design and execution of data migration, validation, and reconciliation processes to support accurate financial and operational reporting. Collaborate with data and technology teams to ensure robust, secure data pipelines between core systems (e.g., ERP, EPM, data warehouse). Cross-Functional Coordination Serve as the bridge between Finance, Operations, Technology, and Data teams to ensure alignment on priorities, requirements, and timelines. Communicate effectively with stakeholders at all levels, translating technical concepts into actionable More ❯

with an organisation that’s passionate about your learning journey and committed to your professional career and personal development. Lead Technology Integration Workstream Drive the financial planning and data integration workstream within a broader transformation or modernization program. Align integration strategy with enterprise financial systems, ensuring data accuracy and consistency across planning, reporting, and forecasting platforms. … maintain a detailed roadmap covering milestones, dependencies, and risk mitigation plans. Oversee day-to-day execution, ensuring timely delivery and adherence to scope, quality, and compliance standards. Drive Data Integration, Migration, and Validation Lead the design and execution of data migration, validation, and reconciliation processes to support accurate financial and operational reporting. Collaborate with data and technology teams to ensure robust, secure data pipelines between core systems (e.g., ERP, EPM, data warehouse). Cross-Functional Coordination Serve as the bridge between Finance, Operations, Technology, and Data teams to ensure alignment on priorities, requirements, and timelines. Communicate effectively with stakeholders at all levels, translating technical concepts into actionable More ❯

with an organisation that’s passionate about your learning journey and committed to your professional career and personal development. Lead Technology Integration Workstream Drive the financial planning and data integration workstream within a broader transformation or modernization program. Align integration strategy with enterprise financial systems, ensuring data accuracy and consistency across planning, reporting, and forecasting platforms. … maintain a detailed roadmap covering milestones, dependencies, and risk mitigation plans. Oversee day-to-day execution, ensuring timely delivery and adherence to scope, quality, and compliance standards. Drive Data Integration, Migration, and Validation Lead the design and execution of data migration, validation, and reconciliation processes to support accurate financial and operational reporting. Collaborate with data and technology teams to ensure robust, secure data pipelines between core systems (e.g., ERP, EPM, data warehouse). Cross-Functional Coordination Serve as the bridge between Finance, Operations, Technology, and Data teams to ensure alignment on priorities, requirements, and timelines. Communicate effectively with stakeholders at all levels, translating technical concepts into actionable More ❯

This team leverages technical expertise in actuarial modelling, statistical analysis, and risk management solutions to support a high-impact team of 90+ underwriters. You will build and maintain data pipelines and data models to elevate the Analytics team and help answer the most difficult business challenges. Key requirements Experience in the London or Lloyd's insurance … markets is essential Have an Advanced degree in Statistics, Computer Science, Mathematics or a related field or relevant industry industry experience Have knowledge of programming and data manipulation skills such as SQL and Python, and version control (Github) is essential Experience building multi-step ETL jobs/data pipelines and working with job scheduling systems is … a plus Demonstrated ability to thrive with minimal oversight and process. You connect the dots across business needs and independently drive projects forward Have strong data visualization skills and experience with Power BI is a bonus Have experience with cloud technologies, preferably Microsoft Azure More ❯

unlock the value of technology and build a more sustainable, more inclusive world. Your Role We are looking for a skilled Spark/Scala Developer to join our data engineering team. The ideal candidate will have hands-on experience in designing, developing, and maintaining large-scale data processing pipelines using Apache Spark and Scala. You will … work closely with data scientists, analysts, and engineers to build efficient data solutions and enable data-driven decision-making. Key Responsibilities: Develop, optimize, and maintain data pipelines and ETL processes using Apache Spark and Scala. Design scalable and robust data processing solutions for batch and real-time data. Collaborate with … cross-functional teams to gather requirements and translate them into technical specifications. Perform data ingestion, transformation, and cleansing from various structured and unstructured sources. Monitor and troubleshoot Spark jobs, ensuring high performance and reliability. Write clean, maintainable, and well-documented code. Participate in code reviews, design discussions, and agile ceremonies. Implement data quality and governance best More ❯

unlock the value of technology and build a more sustainable, more inclusive world. Your Role We are looking for a skilled Spark/Scala Developer to join our data engineering team. The ideal candidate will have hands-on experience in designing, developing, and maintaining large-scale data processing pipelines using Apache Spark and Scala. You will … work closely with data scientists, analysts, and engineers to build efficient data solutions and enable data-driven decision-making. Key Responsibilities: Develop, optimize, and maintain data pipelines and ETL processes using Apache Spark and Scala. Design scalable and robust data processing solutions for batch and real-time data. Collaborate with … cross-functional teams to gather requirements and translate them into technical specifications. Perform data ingestion, transformation, and cleansing from various structured and unstructured sources. Monitor and troubleshoot Spark jobs, ensuring high performance and reliability. Write clean, maintainable, and well-documented code. Participate in code reviews, design discussions, and agile ceremonies. Implement data quality and governance best More ❯

unlock the value of technology and build a more sustainable, more inclusive world. Your Role We are looking for a skilled Spark/Scala Developer to join our data engineering team. The ideal candidate will have hands-on experience in designing, developing, and maintaining large-scale data processing pipelines using Apache Spark and Scala. You will … work closely with data scientists, analysts, and engineers to build efficient data solutions and enable data-driven decision-making. Key Responsibilities: Develop, optimize, and maintain data pipelines and ETL processes using Apache Spark and Scala. Design scalable and robust data processing solutions for batch and real-time data. Collaborate with … cross-functional teams to gather requirements and translate them into technical specifications. Perform data ingestion, transformation, and cleansing from various structured and unstructured sources. Monitor and troubleshoot Spark jobs, ensuring high performance and reliability. Write clean, maintainable, and well-documented code. Participate in code reviews, design discussions, and agile ceremonies. Implement data quality and governance best More ❯

unlock the value of technology and build a more sustainable, more inclusive world. Your Role We are looking for a skilled Spark/Scala Developer to join our data engineering team. The ideal candidate will have hands-on experience in designing, developing, and maintaining large-scale data processing pipelines using Apache Spark and Scala. You will … work closely with data scientists, analysts, and engineers to build efficient data solutions and enable data-driven decision-making. Key Responsibilities: Develop, optimize, and maintain data pipelines and ETL processes using Apache Spark and Scala. Design scalable and robust data processing solutions for batch and real-time data. Collaborate with … cross-functional teams to gather requirements and translate them into technical specifications. Perform data ingestion, transformation, and cleansing from various structured and unstructured sources. Monitor and troubleshoot Spark jobs, ensuring high performance and reliability. Write clean, maintainable, and well-documented code. Participate in code reviews, design discussions, and agile ceremonies. Implement data quality and governance best More ❯

opportunity to take on a high-impact, hands-on leadership role within a scaling global SaaS company that’s using AI to reinvent how internal business systems and data work together. As a Staff Software Engineer (AI/ML) , you’ll be the technical lead for a newly formed AI Engineering function , designing and delivering the internal AI … and data infrastructure that supports smarter decision-making across the organisation. You’ll build and scale AI-powered backend systems that connect enterprise data platforms (CRM, analytics, and communication tools) with large language models — creating intelligent tools and insights that enhance the way teams operate. This is a role with real influence: you’ll own the … adopted across the company. What You’ll Do Lead the design, architecture, and implementation of internal AI/ML systems for business intelligence. Build robust APIs, microservices, and data pipelines to power intelligent, data-driven tools. Develop retrieval-augmented generation (RAG) systems using vector databases for contextual AI. Set the technical direction for backend and AI More ❯

opportunity to take on a high-impact, hands-on leadership role within a scaling global SaaS company that’s using AI to reinvent how internal business systems and data work together. As a Staff Software Engineer (AI/ML) , you’ll be the technical lead for a newly formed AI Engineering function , designing and delivering the internal AI … and data infrastructure that supports smarter decision-making across the organisation. You’ll build and scale AI-powered backend systems that connect enterprise data platforms (CRM, analytics, and communication tools) with large language models — creating intelligent tools and insights that enhance the way teams operate. This is a role with real influence: you’ll own the … adopted across the company. What You’ll Do Lead the design, architecture, and implementation of internal AI/ML systems for business intelligence. Build robust APIs, microservices, and data pipelines to power intelligent, data-driven tools. Develop retrieval-augmented generation (RAG) systems using vector databases for contextual AI. Set the technical direction for backend and AI More ❯

opportunity to take on a high-impact, hands-on leadership role within a scaling global SaaS company that’s using AI to reinvent how internal business systems and data work together. As a Staff Software Engineer (AI/ML) , you’ll be the technical lead for a newly formed AI Engineering function , designing and delivering the internal AI … and data infrastructure that supports smarter decision-making across the organisation. You’ll build and scale AI-powered backend systems that connect enterprise data platforms (CRM, analytics, and communication tools) with large language models — creating intelligent tools and insights that enhance the way teams operate. This is a role with real influence: you’ll own the … adopted across the company. What You’ll Do Lead the design, architecture, and implementation of internal AI/ML systems for business intelligence. Build robust APIs, microservices, and data pipelines to power intelligent, data-driven tools. Develop retrieval-augmented generation (RAG) systems using vector databases for contextual AI. Set the technical direction for backend and AI More ❯

opportunity to take on a high-impact, hands-on leadership role within a scaling global SaaS company that’s using AI to reinvent how internal business systems and data work together. As a Staff Software Engineer (AI/ML) , you’ll be the technical lead for a newly formed AI Engineering function , designing and delivering the internal AI … and data infrastructure that supports smarter decision-making across the organisation. You’ll build and scale AI-powered backend systems that connect enterprise data platforms (CRM, analytics, and communication tools) with large language models — creating intelligent tools and insights that enhance the way teams operate. This is a role with real influence: you’ll own the … adopted across the company. What You’ll Do Lead the design, architecture, and implementation of internal AI/ML systems for business intelligence. Build robust APIs, microservices, and data pipelines to power intelligent, data-driven tools. Develop retrieval-augmented generation (RAG) systems using vector databases for contextual AI. Set the technical direction for backend and AI More ❯

world’s leading organizations unlock the value of technology and build a more sustainable, more inclusive world. YOUR ROLE We are seeking a skilled and hands-on AWS Data Engineer with strong coding expertise and deep experience in building scalable data solutions using AWS services. The ideal candidate will have a solid background in data engineering, Python development, and cloud-native architecture. YOUR PROFILE Design, develop, and maintain robust data pipelines and ETL workflows using AWS services. Implement scalable data processing solutions using PySpark and AWS Glue. Build and manage infrastructure as code using CloudFormation. Develop and deploy serverless applications using AWS Lambda, Step Functions, and S3. Perform data querying and analysis using Athena. Collaborate with Data Scientists to operationalize models using SageMaker. Ensure secure and compliant data handling using IAM, KMS, and VPC configurations. Containerize applications using ECS for scalable deployment. Write clean, testable code in Python, with a strong emphasis on unit testing. Use GitLab for version control, CI/CD More ❯

world’s leading organizations unlock the value of technology and build a more sustainable, more inclusive world. YOUR ROLE We are seeking a skilled and hands-on AWS Data Engineer with strong coding expertise and deep experience in building scalable data solutions using AWS services. The ideal candidate will have a solid background in data engineering, Python development, and cloud-native architecture. YOUR PROFILE Design, develop, and maintain robust data pipelines and ETL workflows using AWS services. Implement scalable data processing solutions using PySpark and AWS Glue. Build and manage infrastructure as code using CloudFormation. Develop and deploy serverless applications using AWS Lambda, Step Functions, and S3. Perform data querying and analysis using Athena. Collaborate with Data Scientists to operationalize models using SageMaker. Ensure secure and compliant data handling using IAM, KMS, and VPC configurations. Containerize applications using ECS for scalable deployment. Write clean, testable code in Python, with a strong emphasis on unit testing. Use GitLab for version control, CI/CD More ❯

Capital Markets Quality Engineer – Safeguard the Integrity of Trading Data Join our Data and Reporting Platform Delivery team and play a critical role in ensuring the reliability, accuracy, and compliance of our capital markets systems. If you're passionate about automated testing, data quality, and continuous delivery, this is your opportunity to shape how … be embedded in one of our Capital Markets stream-aligned teams within the Retirement Solutions and Asset Management division. You'll design and implement automated testing frameworks, validate data accuracy across systems, and embed quality practices throughout the development lifecycle. Your work will directly support trading workflows, OMS integrations, and regulatory compliance. Key Responsibilities Develop and maintain automated … test frameworks for data pipelines, APIs, and integrations. Build automated tests for data accuracy, transformations, and trading workflows across OMS platforms (e.g., Aladdin, Murex, Charles River). Validate integration points between trading systems and data platforms. Support regression, performance, and functional testing. Embed testing early in the delivery lifecycle and contribute to CI/ More ❯

Senior Software Engineer (Data-Centric) Manchester or South London (Hybrid 2 x a week in office) 70,000 85,000 + 10% bonus + excellent benefits Our client is a global self-defence intelligence and technology organisation helping governments and defence agencies act with confidence. They use AI, graph databases, cloud, and search technologies to deliver faster, smarter … services used globally. Develop in one or more core technology areas: Graph databases (GraphDB, Neo4j) Search technologies (ElasticSearch, OpenSearch, SOLR) ML/AI frameworks (SageMaker, Bedrock) Cloud-based data engineering (AWS) Build and maintain cloud-native data pipelines, ensuring data quality, accuracy, and reliability. 5+ years experience in software or backend engineering. Proven ability … to deliver reliable, scalable systems using cloud technologies (AWS preferred). Experience with complex data systems - structured, unstructured, or graph-based. Work on advanced data and intelligence platforms that have global impact. Join a modern, agile technology organisation where innovation is encouraged and rewarded. Help shape the next generation of open-source intelligence technology. More ❯

Up to 125,000 base + bonus & benefits A leading global consulting and analytics firm is seeking a Databricks Architect to drive architecture and delivery of large-scale data solutions for their commercial insurance clients. This is a senior role, shaping data strategy and modernising platforms that underpin underwriting, claims, and pricing. Key Responsibilities Lead end … to-end architecture and solution design across Databricks and Azure cloud ecosystems. Build scalable lakehouse architectures, advanced analytics, and AI/ML data pipelines. Work closely with insurance stakeholders to translate business needs into technical solutions. Ensure best practices in data governance, security, and regulatory compliance. Provide technical leadership to engineering teams and contribute to architecture … standards. Skills & Experience Proven experience as a Data/Solution Architect with strong expertise in Databricks (Delta Lake, MLflow, Spark) . Deep understanding of commercial insurance processes (underwriting, claims, personal/specialty lines). Strong background in Azure (Synapse, Data Factory, Fabric) or similar cloud data platforms. Knowledge of data governance, integration More ❯