276 to 300 of 399 Data Pipeline Jobs in England

A small and highly successful organisation in the sustainability space are seeking a Data Engineer with some Power Platform experience to join their team. They have an office space in London, though this role is remote and is therefore open to candidates across the UK. They are on a mission to help businesses to reduce their carbon emissions … ecosystem - including Azure, Power Apps, Power Pages and Power BI, and allows for real-time engagement and AI-guided action plans for carbon reduction. Responsibilities include: Use Azure Data Factory to ingest, transform, and expose data across the platform Help centralise data into a Data Lake, ensuring it's clean, structured, and … accessible Build data pipelines for LLMs and integrate tools like GPT to drive intelligent outputs Develop user-facing applications using Power Apps and Power Pages Design and manage APIs to connect front-end applications and integrate external services (e.g. procurement systems, AI tools) Collaborate directly with users to turn complex needs into streamlined solutions It's a broad More ❯

A small and highly successful organisation in the sustainability space are seeking a Data Engineer with some Power Platform experience to join their team. They have an office space in London, though this role is remote and is therefore open to candidates across the UK. They are on a mission to help businesses to reduce their carbon emissions … ecosystem - including Azure, Power Apps, Power Pages and Power BI, and allows for real-time engagement and AI-guided action plans for carbon reduction. Responsibilities include: Use Azure Data Factory to ingest, transform, and expose data across the platform Help centralise data into a Data Lake, ensuring it's clean, structured, and … accessible Build data pipelines for LLMs and integrate tools like GPT to drive intelligent outputs Develop user-facing applications using Power Apps and Power Pages Design and manage APIs to connect front-end applications and integrate external services (e.g. procurement systems, AI tools) Collaborate directly with users to turn complex needs into streamlined solutions It's a broad More ❯

A small and highly successful organisation in the sustainability space are seeking a Data Engineer with some Power Platform experience to join their team. They have an office space in London, though this role is remote and is therefore open to candidates across the UK. They are on a mission to help businesses to reduce their carbon emissions … ecosystem - including Azure, Power Apps, Power Pages and Power BI, and allows for real-time engagement and AI-guided action plans for carbon reduction. Responsibilities include: Use Azure Data Factory to ingest, transform, and expose data across the platform Help centralise data into a Data Lake, ensuring it's clean, structured, and … accessible Build data pipelines for LLMs and integrate tools like GPT to drive intelligent outputs Develop user-facing applications using Power Apps and Power Pages Design and manage APIs to connect front-end applications and integrate external services (e.g. procurement systems, AI tools) Collaborate directly with users to turn complex needs into streamlined solutions It's a broad More ❯

A small and highly successful organisation in the sustainability space are seeking a Data Engineer with some Power Platform experience to join their team. They have an office space in London, though this role is remote and is therefore open to candidates across the UK. They are on a mission to help businesses to reduce their carbon emissions … ecosystem - including Azure, Power Apps, Power Pages and Power BI, and allows for real-time engagement and AI-guided action plans for carbon reduction. Responsibilities include: Use Azure Data Factory to ingest, transform, and expose data across the platform Help centralise data into a Data Lake, ensuring it's clean, structured, and … accessible Build data pipelines for LLMs and integrate tools like GPT to drive intelligent outputs Develop user-facing applications using Power Apps and Power Pages Design and manage APIs to connect front-end applications and integrate external services (e.g. procurement systems, AI tools) Collaborate directly with users to turn complex needs into streamlined solutions It's a broad More ❯

Data Engineer (Defence) £45,000 - 55,000 GBP Onsite WORKING Location: Stevenage, Cambridge, East of England - United Kingdom Type: Permanent Data Engineer - Generative AI | Defence Technology A leading international defence organisation is seeking an experienced Data Engineer with a focus on Generative AI to join their IM GenAI Delivery Office in the UK. You … support advanced defence systems and critical internal projects. What you'll do: Develop and maintain robust, secure, and high-integrity datasets for internal defence applications. Design and optimise data pipelines to ensure reliable, compliant, and scalable solutions. Apply your expertise in Generative AI, NLP, and OCR to enhance and automate data processes. Build ETL workflows, APIs … and other data integration solutions tailored for mission-critical environments . Provide guidance on emerging technologies and support the roadmap for next-generation defence data solutions . Skills and experience: Strong knowledge of SQL (MS SQL, Oracle) and noSQL databases (e.g., MongoDB, Neo4J, InfluxDB). Hands-on experience with data processing and integration tools More ❯

Lead Data Engineer- £70,000 Industry: Banking and Financial Services Location : Flexible- Basingstoke (3 Days a Week) Overview of Role: This role will lead the design and optimisation of our Azure Databricks data platform , ensuring it is secure, scalable, and well governed. You'll take ownership of change controls, lineage, and documentation while driving adoption of … best practice. Alongside this, you'll support AI/ML initiatives and provide technical guidance to colleagues, driving value from data. Key Responsibilities Lead the roadmap for data engineering, ensuring alignment with business needs and regulatory requirements. Own the design and optimisation of the Azure Databricks platform, focusing on performance, resilience, and compliance. Implement change controls, lineage, and … observability to detect and resolve issues. Maintain clear documentation of data pipelines, models, and governance processes. Implement tools to improve data quality, efficiency, and adoption. Support AI/ML delivery by building/optimising, ensuring platform readiness. Manage SQL workloads within Databricks, balancing cost and performance Ensure access controls, GDPR compliance, and security standards Work with More ❯

Data Scientist (LLMs & ML) - UK Remote A fast-growing healthcare organization is seeking a Data Scientist with strong experience in machine learning, deep learning, and Large Language Models (LLMs) to help drive innovation and automation in clinical services. This is a full-time, remote UK-based role. ***Unfortunately we can't consider candidates that require sponsorship … solutions (e.g., LangChain, Mastra.ai, Pydantic) for document processing, summarization, and clinical Q&A systems. Develop and optimize predictive models using scikit-learn, PyTorch, TensorFlow, and XGBoost. Design robust data pipelines using tools like Spark and Kafka for real-time and batch processing. Manage ML lifecycle with tools such as Databricks , MLflow , and cloud-native platforms (Azure preferred). … Collaborate with engineering teams to ensure scalable, secure ML infrastructure aligned with compliance standards (e.g., ISO27001). Ensure data governance, particularly around sensitive healthcare data. Share best practices and stay current with developments in AI, ML, and LLMs. Proven experience with LLM frameworks and transformer-based architectures. Strong Python skills and familiarity with key ML/DL libraries. More ❯

team alignment. Nice to Have Experience with orchestration frameworks ( LangChain, LangGraph ) and multi-agent workflows Knowledge of vector databases, RAG pipelines, and lightweight model hosting Ability to design data pipelines and feedback loops for improving AI-driven features Awareness of emerging AI areas such as multimodal, edge AI, or AI in DevOps/observability Why Join? A leadership More ❯

technology and desire to mentor a team. You will hold proven experience of architecting solutions using Python (FastAPI/FlasK) and will lead the technical designs for APIs, data pipelines, cloud infrastructure and backend orchestration. You will also have experience with AWS, while familiarity with MongoDB would be advantageous to your application. In this role of significant responsibility More ❯

technology and ability to lead a team. You will hold proven experience of architecting solutions using Python (FastAPI/FlasK) and will lead the technical designs for APIs, data pipelines, cloud infrastructure and backend orchestration. You will also have experience with AWS, while familiarity with MongoDB would be advantageous to your application. In this role of significant responsibility More ❯

technology and desire to mentor a team. You will hold proven experience of architecting solutions using Python (FastAPI/FlasK) and will lead the technical designs for APIs, data pipelines, cloud infrastructure and backend orchestration. You will also have experience with AWS, while familiarity with MongoDB would be advantageous to your application. In this role of significant responsibility More ❯

Nice to Have Experience with orchestration frameworks (e.g., LangChain, LangGraph) and multi-agent workflows. Familiarity with vector databases, RAG pipelines, and lightweight model hosting. Ability to set up data pipelines and feedback loops for improving AI-driven features. Awareness of emerging areas such as multimodal AI, edge AI, and AI applications in DevOps/observability. The Ideal Candidate More ❯

As an FPGA/Digital Design Engineer, you will: Architect, design, implement, verify, and integrate digital logic (RTL) for FPGA/SoC platforms Work on signal processing, communications, data pipelines, digital beamforming, or related SDR blocks Develop testbench environments, simulations, and formal/verifiable test strategies Perform timing closure, clock domain crossing, resource balancing, power optimisation Define interfaces More ❯

BluAge modernization tools, Java Spring Boot, and Apache Kafka. The ideal candidate will play a key role in modernizing legacy applications, building scalable microservices, and implementing real time data pipelines. Must have technical skills: AWS BluAge Kafka Java Springboot Nice to have skills: Good to have some knowledge on banking domain Responsibilities: Solution design To Involve in all More ❯

Lead Data Engineer - £65-70k + bonus + benefits - Basingstoke 3 days a week The Opportunity: We are looking for a Lead Data Engineer to join a Basingstoke based organisation who are investing heavily in their Digital Transformation Programme. The Role: There is currently 1 Data Engineer in the team and you'll … join to play a proactive role in the delivery of their next-generation data platform, manage/mentor the existing person and drive the design, development and governance of the data pipelines. You'll be working really closely with stakeholders across the technology function and within the business and will the availability, integrity and compliance of … background/attributes include: - In depth experience of modern data solution architecture design and delivery in a hybrid cloud environment but predominantly Azure/Databricks - Experience of pipeline orchestration management - Strong exprerience with Databricks - Experience of implementing machine learning and AI - Tooling such as Purview and Unity Catalog as well as the use of observability tools such More ❯

Genome Campus near Cambridge, UK. The Chemical Biology Resources team provides world-leading chemogenomics resources to the scientific community. ChEMBL is a database of quantitative small-molecule bioactivity data curated primarily from the scientific literature widely used to support drug discovery projects in industry and academia. SureChEMBL is a complementary patent resource containing chemical structures extracted from patents … of biological interest. OPSIN converts systematic chemical names into chemical structures. In this role, you will develop and maintain software infrastructure to ensure robust, reliable, and automated ChEMBL data releases through streamlined build, packaging, testing, and deployment processes, while also assisting in the development and maintenance of other services supported by the team. You will be working closely … mix of professional scientists and software engineers. The EBI is part of the European Molecular Biology Laboratory (EMBL) and it is a world-leading bioinformatics centre providing biological data to the scientific community with expertise in data storage, analysis and representation. EMBL-EBI provides freely available data from life science experiments, performs basic research More ❯

Data Engineer - AI & Machine Learning Focus Location: Kings Cross, London Hybrid Working: 2-3 days in the office, remainder from home Duration: 6 Months Contract Type: Inside IR35 (via Umbrella) Are you passionate about building data solutions that drive cutting-edge AI and Machine Learning applications? We're looking for a talented Data Engineer … to join a forward-thinking team developing transformative therapies for both existing and emerging diseases. This is your opportunity to work on impactful projects at the intersection of data engineering and life sciences, using modern cloud technologies to deliver scalable, reliable data pipelines that support scientific innovation. What You'll Be Doing: Designing and building data pipelines using tools like Python, Spark, SQL, BigQuery, and Google Cloud Storage Ensuring pipelines meet the specific needs of data-driven scientific applications Writing high-quality, well-documented code with automated testing Monitoring and improving key metrics across tools and services Collaborating with cross-functional teams to build end-to-end data solutions Participating More ❯

smooth and efficient functioning and development of our databases and pipelines. You will be responsible for developing, optimising and managing these databases and ensuring a high-level of data quality and integrity. You will be defining the principles and processes that underpin data management while ensuring we maintain the highest standards for data management … repeatable and scalable. Create and manage database schemas, tables, stored procedures, indexes, and triggers. Ensuring high availability of databases through clustering, replication, and other techniques. Implement and maintain data pipelines and ETL processes for real-time and batch data ingestion. Monitor database performance and proactively address issues related to tuning and optimization. Ensure data security, privacy, integrity, and compliance with internal and external regulations. Collaborate with third-party suppliers, software developers, data analysts, DevOps, and other stakeholders to meet project requirements. Plan and implement database backups, restores, and disaster recovery strategies. Taking an active role in supporting and in finding the root cause of operational issues and preventing recurrences. Support More ❯

smooth and efficient functioning and development of our databases and pipelines. You will be responsible for developing, optimising and managing these databases and ensuring a high-level of data quality and integrity. You will be defining the principles and processes that underpin data management while ensuring we maintain the highest standards for data management … repeatable and scalable. Create and manage database schemas, tables, stored procedures, indexes, and triggers. Ensuring high availability of databases through clustering, replication, and other techniques. Implement and maintain data pipelines and ETL processes for real-time and batch data ingestion. Monitor database performance and proactively address issues related to tuning and optimization. Ensure data security, privacy, integrity, and compliance with internal and external regulations. Collaborate with third-party suppliers, software developers, data analysts, DevOps, and other stakeholders to meet project requirements. Plan and implement database backups, restores, and disaster recovery strategies. Taking an active role in supporting and in finding the root cause of operational issues and preventing recurrences. Support More ❯

You will join a successful international insurance company who specialise in commercial insurance and reinsurance products. My client is looking for an experienced Senior Data Engineer to play a pivotal role in designing and implementing next-generation data warehousing solutions, with a focus on Snowflake and AWS technologies. This is a fantastic opportunity to shape the … future of data architecture and drive innovation in a dynamic, collaborative environment. Main Responsibilities: Lead the modernisation journey to Snowflake-based data warehousing solutions within an AWS environment. Design and implement scalable, secure, and cost-efficient data solutions using AWS services such as Glue, Lambda, S3, Redshift, and Step Functions. Develop robust data pipelines and analytics platforms, ensuring high availability, performance, and maintainability. Drive automation and CI/CD practices for data workflows, ensuring reliable deployment and monitoring. Mentor and support team members through code reviews, pair programming, and knowledge sharing. Collaborate with stakeholders, product owners, and developers to ensure alignment between requirements and data solutions. Benefits More ❯

Retention Data | BI Analyst 12-month contract day rate of £230-£255 (umbrella) Are you passionate about turning data into insights that drive customer loyalty? We are looking for a skilled BI Analyst on a 12-month contract to work on high-impact projects focused on customer retention . This role offers a You will transform … you will uncover actionable insights and drive retention-focused initiatives. Key Responsibilities Build automated, scalable reporting solutions focused on customer retention . Create dashboards combining internal and external data sources for senior leadership presentations. Collaborate with teams to identify gaps in data capture strategies and implement enhancements. Provide actionable insights on key retention metrics. Maintain relationships … with stakeholders to ensure retention-related data needs are met. Qualifications Proven experience as a BI Analyst or similar role, ideally focused on customer retention . Strong ability to analyze and combine large datasets using advanced SQL . Experience with BI/reporting tools such as Tableau or Qlik . Experience building and maintaining data pipelines More ❯

Retention Data | BI Analyst 12-month contract day rate of £230-£255 (umbrella) Are you passionate about turning data into insights that drive customer loyalty? We are looking for a skilled BI Analyst on a 12-month contract to work on high-impact projects focused on customer retention . This role offers a You will transform … you will uncover actionable insights and drive retention-focused initiatives. Key Responsibilities Build automated, scalable reporting solutions focused on customer retention . Create dashboards combining internal and external data sources for senior leadership presentations. Collaborate with teams to identify gaps in data capture strategies and implement enhancements. Provide actionable insights on key retention metrics. Maintain relationships … with stakeholders to ensure retention-related data needs are met. Qualifications Proven experience as a BI Analyst or similar role, ideally focused on customer retention . Strong ability to analyze and combine large datasets using advanced SQL . Experience with BI/reporting tools such as Tableau or Qlik . Experience building and maintaining data pipelines More ❯

the Platform & AI Enablement team within our GPTO Engineering organization, reporting to the Sr. Director Engineering API. The team is responsible for building core capabilities that power our data and AI infrastructure-enabling scalable, resilient, and intelligent systems to support enterprise-scale Business Planning Software solutions. Your Impact Design, build, and operate platform capabilities supporting batch, streaming, and … systems using Apache Kafka, Flink, Pulsar, and cloud-native technologies Collaborate with AI/ML teams to deploy models and enable generative AI use cases Implement integrations with data lakes and event stores to enable real-time data flow Contribute fullstack features, including React-based frontends and backend services in Java and Python Ensure code quality … software engineering, especially in platform/backend systems Expert-level skills in Java and strong proficiency in Python Experience with Apache Kafka, Flink, and Pulsar for building distributed data pipelines Familiarity with scalable data storage and data lake integrations Proven ability to integrate AI/ML models and work with prompt-based applications Strong More ❯

using NodeJS/Node-RED and Microsoft SQL Server. This role includes building real-time monitoring dashboards to support critical business workflows, streamline approval processes, and facilitate accurate data reconciliation. A strong emphasis is placed on creating scalable, testable integration pipelines with effective error management, high availability, and full lifecycle testing to ensure robust API interactions and telemetry … systems is a plus Working knowledge of Microsoft Azure services is desirable Key Responsibilities: Develop and manage SQL Server databases used for API-driven integrations and data pipeline control Build technology-agnostic automation workflows incorporating detailed orchestration and exception handling Connect and synchronise disparate applications using RESTful APIs and integration middleware to ensure secure, reliable communication Establish More ❯

Data Analytics Engineer My client in the Insurance industry are searching for a Data Analytics Engineer. We are looking for someone who has: A strong engineering background and possibly BI experience. Solid proficiency in SQL and Python as core programming skills. Proven experience in building data pipelines, including API integrations. Knowledge of Power BI … and Databricks. Key responsibilities will include: Supporting BAU (Business-as-Usual) tasks. Leading a major project to rebuild two core APIs and their associated pipelines. Assisting in optimising several pipeline SQl based. Assisting in optimising a few Power BI semantic models. Please apply! Carbon60, Lorien & SRG - The Impellam Group STEM Portfolio are acting as an Employment Business in relation More ❯