1,001 to 1,025 of 1,208 Data Pipeline Jobs in the UK excluding London

Databricks Tech Lead – Azure Data Engineering (Contract) We are looking for a highly technical, hands-on Databricks Tech Lead to join a large-scale digital transformation programme within the Energy Trading space. This role will focus on building, optimising, and scaling a data lake on Azure with particular emphasis on streaming pipelines, Delta Live Tables (DLT … and cost-efficient architecture. Role: Designing and developing streaming and batch data pipelines in Databricks (including DLT). Managing structured streaming in continuous mode, ensuring low-latency data delivery. Collaborating with architects, consultants, and business stakeholders to build a unified Azure-based lakehouse. Troubleshooting and optimising performance bottlenecks in high-volume streaming systems. Applying best practices … for cloud cost management and efficient Databricks resource usage. Integrating data from on-prem and cloud sources using tools like Qlik Replicate (CDC). Supporting future phases involving data science and machine learning enablement. Tech Environment Platform: Azure Core Tools: Databricks, Delta Live Tables, Qlik Replicate (CDC) Architecture: Hybrid (on-prem + cloud) Data More ❯

Databricks Tech Lead – Azure Data Engineering (Contract) We are looking for a highly technical, hands-on Databricks Tech Lead to join a large-scale digital transformation programme within the Energy Trading space. This role will focus on building, optimising, and scaling a data lake on Azure with particular emphasis on streaming pipelines, Delta Live Tables (DLT … and cost-efficient architecture. Role: Designing and developing streaming and batch data pipelines in Databricks (including DLT). Managing structured streaming in continuous mode, ensuring low-latency data delivery. Collaborating with architects, consultants, and business stakeholders to build a unified Azure-based lakehouse. Troubleshooting and optimising performance bottlenecks in high-volume streaming systems. Applying best practices … for cloud cost management and efficient Databricks resource usage. Integrating data from on-prem and cloud sources using tools like Qlik Replicate (CDC). Supporting future phases involving data science and machine learning enablement. Tech Environment Platform: Azure Core Tools: Databricks, Delta Live Tables, Qlik Replicate (CDC) Architecture: Hybrid (on-prem + cloud) Data More ❯

will be joining a community of Graduates and Apprentices across the business. On joining, you will be assigned to start in one of the following roles. Insights and Data (I&D) Build secure, scalable data pipelines using modern tools and cloud platforms. Create dashboards, automate testing, and support data governance. Work on cloud migration … by its clients to address the entire breadth of their business needs, from strategy and design to operations, fueled by the fast evolving and innovative world of cloud, data, AI, connectivity, software, digital engineering and platforms. Get the Future You Want More ❯

will be joining a community of Graduates and Apprentices across the business. On joining, you will be assigned to start in one of the following roles. Insights and Data (I&D) Build secure, scalable data pipelines using modern tools and cloud platforms. Create dashboards, automate testing, and support data governance. Work on cloud migration … by its clients to address the entire breadth of their business needs, from strategy and design to operations, fueled by the fast evolving and innovative world of cloud, data, AI, connectivity, software, digital engineering and platforms. Get the Future You Want More ❯

Overview VP of AI/ML/Data - UK (Hybrid role with offices in South Wales & London). Highly Competitive (circa £100,000 - £150,000) + Bonus + Equity. What You'll Do Step into a senior leadership role with a mission to drive innovation across an AI-powered SaaS platform. As VP of AI, you'll shape … just experimentation-so your impact will be visible, measurable, and far-reaching. You'll also play a key role in modernising and streamlining the platform's AI and data infrastructure, championing MLOps best practices, and aligning technical delivery with strategic goals. What You'll Bring Significant experience leading AI/ML teams in fast-paced, SaaS-driven environments … Proven ability to deliver AI capabilities in production, including model training, deployment, monitoring, and lifecycle management Strong technical background across machine learning, data pipelines, and cloud platforms (e.g. Snowflake, Databricks, Azure) Exceptional communication and influencing skills, with the ability to align executive stakeholders around AI strategy A pragmatic, delivery-focused mindset and passion for scalable, secure, and privacy More ❯

shape our culture and guide our decision-making. We are committed to: Excellence Responsibility Empowerment Connectivity Courage Role Overview We are seeking a seasoned and visionary Lead, AI & Data Science to drive the strategy, development, and deployment of advanced AI solutions across our Digital Solutions department. This role is leadership and technically grounded, you will define and guide … and the ability to review, advise, and step in when critical decisions or mentorship are needed. You will play a pivotal role in building and scaling the AI & Data Science team, shaping career paths, and representing the department in global events, partnerships, and thought leadership platforms. Key Responsibilities Strategic & Technical Leadership Define and lead the AI & Data … CI/CD for ML, monitoring, retraining, governance). Drive innovation while balancing pragmatism and production readiness . Mentorship & Team Development Build and grow a world-class AI & Data Science team , including hiring, onboarding, and performance management . Mentor and coach team members to elevate technical depth and problem-solving skills. Create career development plans, learning paths, and More ❯

shape our culture and guide our decision-making. We are committed to: Excellence Responsibility Empowerment Connectivity Courage Role Overview We are seeking a seasoned and visionary Lead, AI & Data Science to drive the strategy, development, and deployment of advanced AI solutions across our Digital Solutions department. This role is leadership and technically grounded, you will define and guide … and the ability to review, advise, and step in when critical decisions or mentorship are needed. You will play a pivotal role in building and scaling the AI & Data Science team, shaping career paths, and representing the department in global events, partnerships, and thought leadership platforms. Key Responsibilities Strategic & Technical Leadership Define and lead the AI & Data … CI/CD for ML, monitoring, retraining, governance). Drive innovation while balancing pragmatism and production readiness . Mentorship & Team Development Build and grow a world-class AI & Data Science team , including hiring, onboarding, and performance management . Mentor and coach team members to elevate technical depth and problem-solving skills. Create career development plans, learning paths, and More ❯

The Role As AI Reporting & Insights Lead you'll own the intelligence layer of our AI-powered platform—the dashboards, reports, prompts, and templates that transform raw customer data into clarity. This is a unique hybrid role combining AI expertise, analytical rigor, and customer-facing delivery. You'll define what "great AI-powered reporting" looks like and make … features Close the feedback loop: Channel customer needs back to Product to shape the reporting roadmap Product Collaboration & Innovation (20%) Shape AI reporting features: Partner with Product and Data Science to spec new capabilities—structured output schemas, prompt templates, embedding-based discovery Experiment & prototype: Build proof-of-concepts for new report types using Python, pandas, OpenAI APIs, and … our data pipelines QA & iteration: Test new features, validate AI outputs for accuracy and relevance, ensure quality before launch Operations & Optimization (10%) Monitor report health: Track delivery, engagement, and quality metrics; debug when outputs degrade Performance tuning: Optimize report generation costs and latency; balance API usage with quality Document everything: Maintain clear documentation for prompts, templates, and best More ❯

The Role As AI Reporting & Insights Lead you'll own the intelligence layer of our AI-powered platform—the dashboards, reports, prompts, and templates that transform raw customer data into clarity. This is a unique hybrid role combining AI expertise, analytical rigor, and customer-facing delivery. You'll define what "great AI-powered reporting" looks like and make … features Close the feedback loop: Channel customer needs back to Product to shape the reporting roadmap Product Collaboration & Innovation (20%) Shape AI reporting features: Partner with Product and Data Science to spec new capabilities—structured output schemas, prompt templates, embedding-based discovery Experiment & prototype: Build proof-of-concepts for new report types using Python, pandas, OpenAI APIs, and … our data pipelines QA & iteration: Test new features, validate AI outputs for accuracy and relevance, ensure quality before launch Operations & Optimization (10%) Monitor report health: Track delivery, engagement, and quality metrics; debug when outputs degrade Performance tuning: Optimize report generation costs and latency; balance API usage with quality Document everything: Maintain clear documentation for prompts, templates, and best More ❯

The Role As AI Reporting & Insights Lead you'll own the intelligence layer of our AI-powered platform—the dashboards, reports, prompts, and templates that transform raw customer data into clarity. This is a unique hybrid role combining AI expertise, analytical rigor, and customer-facing delivery. You'll define what "great AI-powered reporting" looks like and make … features Close the feedback loop: Channel customer needs back to Product to shape the reporting roadmap Product Collaboration & Innovation (20%) Shape AI reporting features: Partner with Product and Data Science to spec new capabilities—structured output schemas, prompt templates, embedding-based discovery Experiment & prototype: Build proof-of-concepts for new report types using Python, pandas, OpenAI APIs, and … our data pipelines QA & iteration: Test new features, validate AI outputs for accuracy and relevance, ensure quality before launch Operations & Optimization (10%) Monitor report health: Track delivery, engagement, and quality metrics; debug when outputs degrade Performance tuning: Optimize report generation costs and latency; balance API usage with quality Document everything: Maintain clear documentation for prompts, templates, and best More ❯

fraudulent registrations out of 1 million annually — and saving the UK taxpayer hundreds of millions in lost revenue. To enhance these capabilities, the team is migrating and automating data ingestion and risk analytics processes within the SAS Viya 4 platform, leveraging 20+ data sources from the Minerva Oracle estate. This new test role sits at the … heart of that transformation — ensuring that automated testing, data quality, and performance assurance are embedded into every stage of delivery. The Role: We’re looking for a Network Test Specialist/Test Automation Engineer with strong experience in data and analytics platforms, ideally including SAS Viya. You’ll join a newly formed agile team within the … the platform. You’ll own the automation test strategy, build frameworks from the ground up, integrate them into CI/CD pipelines, and ensure that every component — from data pipelines to user interfaces — is validated efficiently and reliably. Key Responsibilities: Define and implement test automation strategies for SAS Viya 4 applications (including Visual Analytics and Intelligent Decisioning). More ❯

fraudulent registrations out of 1 million annually - and saving the UK taxpayer hundreds of millions in lost revenue. To enhance these capabilities, the team is migrating and automating data ingestion and risk analytics processes within the SAS Viya 4 platform, leveraging 20+ data sources from the Minerva Oracle estate. This new test role sits at the … heart of that transformation - ensuring that automated testing, data quality, and performance assurance are embedded into every stage of delivery. The Role: We're looking for a Network Test Specialist/Test Automation Engineer with strong experience in data and analytics platforms, ideally including SAS Viya. You'll join a newly formed agile team within the … the platform. You'll own the automation test strategy, build frameworks from the ground up, integrate them into CI/CD pipelines, and ensure that every component - from data pipelines to user interfaces - is validated efficiently and reliably. Key Responsibilities: Define and implement test automation strategies for SAS Viya 4 applications (including Visual Analytics and Intelligent Decisioning). More ❯

THE COMPANY This organisation is pushing the boundaries of AI and data-driven software, developing next-generation tools that integrate advanced machine learning with scalable backend systems. They're looking for an experienced Python Backend Engineer to support the build and integration of key AI components across their tech stack. THE ROLE As a Contract Python Backend Engineer … you'll take ownership of backend development for a Python-based web application, ensuring smooth integration with AI/ML pipelines and data infrastructure. You'll work closely with data engineers, AI specialists, and frontend developers to deliver robust, production-quality software. Key Responsibilities: Build and maintain Python web applications (FastAPI preferred) Integrate AI/ML … components with backend, data, and compute infrastructure Deliver clean, well-documented code following best practices Implement automated testing frameworks and CI/CD workflows Monitor and improve performance metrics across deployed tools Collaborate with technical teams to develop end-to-end data pipelines YOUR SKILLS AND EXPERIENCE Essential: Proven Python backend development experience (FastAPI, Flask, or More ❯

THE COMPANY This organisation is pushing the boundaries of AI and data-driven software, developing next-generation tools that integrate advanced machine learning with scalable backend systems. They're looking for an experienced Python Backend Engineer to support the build and integration of key AI components across their tech stack. THE ROLE As a Contract Python Backend Engineer … you'll take ownership of backend development for a Python-based web application, ensuring smooth integration with AI/ML pipelines and data infrastructure. You'll work closely with data engineers, AI specialists, and frontend developers to deliver robust, production-quality software. Key Responsibilities: Build and maintain Python web applications (FastAPI preferred) Integrate AI/ML … components with backend, data, and compute infrastructure Deliver clean, well-documented code following best practices Implement automated testing frameworks and CI/CD workflows Monitor and improve performance metrics across deployed tools Collaborate with technical teams to develop end-to-end data pipelines YOUR SKILLS AND EXPERIENCE Essential: Proven Python backend development experience (FastAPI, Flask, or More ❯

table creation, SAS Studio V and SAS RTENG to build the SA registration network, SAS Viya 3.5 tools for risk assessment of new SA registrations. The POC leveraged data from 20 different sources, most of which were already housed in the Minerva Oracle database. Previously, some data had to be transferred manually. However, the automated file … transfers described in this Solution Design Document (SDD) will now move that data to the SAS platform using approved Enterprise Architecture (EA) integration patterns, with the initial phase, targeted for delivery in April 2026. This role will form part of a new scrum team within Minerva Platform to develop and deliver the Ingestion and Risking within the SAS … implement automation test strategies and plans for SAS Viya 4 applications, for UI based Visual Analytics (VA) and Intelligent Decisioning (ID), as well as for integration testing of data pipelines. Develop and maintain test scripts using appropriate automation frameworks (e.g., Selenium, PyTest, JUnit) and integrate them into CI/CD pipelines. Confirm the test data strategy More ❯

table creation, SAS Studio V and SAS RTENG to build the SA registration network, SAS Viya 3.5 tools for risk assessment of new SA registrations. The POC leveraged data from 20 different sources, most of which were already housed in the Minerva Oracle database. Previously, some data had to be transferred manually. However, the automated file … transfers described in this Solution Design Document (SDD) will now move that data to the SAS platform using approved Enterprise Architecture (EA) integration patterns, with the initial phase, targeted for delivery in April 2026. This role will form part of a new scrum team within Minerva Platform to develop and deliver the Ingestion and Risking within the SAS … implement automation test strategies and plans for SAS Viya 4 applications, for UI based Visual Analytics (VA) and Intelligent Decisioning (ID), as well as for integration testing of data pipelines. Develop and maintain test scripts using appropriate automation frameworks (e.g., Selenium, PyTest, JUnit) and integrate them into CI/CD pipelines. Confirm the test data strategy More ❯

About ClearWatt ClearWatt is on a mission to make electric vehicle ownership transparent and accessible. Our platform combines real-world data, telematics, and proprietary analytics to deliver trusted insights into EV battery health, real-world range, and performance. We partner with car dealers, leasing companies, energy companies and consumers to help them understand the true condition of electric … vehicles - unlocking confidence across the EV ecosystem. We’re scaling rapidly, building the UK’s most comprehensive EV data intelligence platform - spanning OBD diagnostics, API-based vehicle integrations, and real-time energy data streams. The Role We’re looking for a skilled Backend Developer with strong experience in C# and the .NET ecosystem to help us … scale and evolve our core data and API infrastructure. You’ll play a central role in designing, building, and optimising services that power our EV testing, reporting, and data-sharing products. This role is primarily backend-focused, but familiarity with modern frontend frameworks (React/React Native/Next.js) is a bonus. Key Responsibilities Develop and More ❯

About ClearWatt ClearWatt is on a mission to make electric vehicle ownership transparent and accessible. Our platform combines real-world data, telematics, and proprietary analytics to deliver trusted insights into EV battery health, real-world range, and performance. We partner with car dealers, leasing companies, energy companies and consumers to help them understand the true condition of electric … vehicles - unlocking confidence across the EV ecosystem. We’re scaling rapidly, building the UK’s most comprehensive EV data intelligence platform - spanning OBD diagnostics, API-based vehicle integrations, and real-time energy data streams. The Role We’re looking for a skilled Backend Developer with strong experience in C# and the .NET ecosystem to help us … scale and evolve our core data and API infrastructure. You’ll play a central role in designing, building, and optimising services that power our EV testing, reporting, and data-sharing products. This role is primarily backend-focused, but familiarity with modern frontend frameworks (React/React Native/Next.js) is a bonus. Key Responsibilities Develop and More ❯

About ClearWatt ClearWatt is on a mission to make electric vehicle ownership transparent and accessible. Our platform combines real-world data, telematics, and proprietary analytics to deliver trusted insights into EV battery health, real-world range, and performance. We partner with car dealers, leasing companies, energy companies and consumers to help them understand the true condition of electric … vehicles - unlocking confidence across the EV ecosystem. We’re scaling rapidly, building the UK’s most comprehensive EV data intelligence platform - spanning OBD diagnostics, API-based vehicle integrations, and real-time energy data streams. The Role We’re looking for a skilled Backend Developer with strong experience in C# and the .NET ecosystem to help us … scale and evolve our core data and API infrastructure. You’ll play a central role in designing, building, and optimising services that power our EV testing, reporting, and data-sharing products. This role is primarily backend-focused, but familiarity with modern frontend frameworks (React/React Native/Next.js) is a bonus. Key Responsibilities Develop and More ❯

table creation, SAS Studio V and SAS RTENG to build the SA registration network, SAS Viya 3.5 tools for risk assessment of new SA registrations. The POC leveraged data from 20 different sources, most of which were already housed in the Minerva Oracle database. Previously, some data had to be transferred manually. However, the automated file … transfers described in this Solution Design Document (SDD) will now move that data to the SAS platform using approved Enterprise Architecture (EA) integration patterns, with the initial phase, targeted for delivery in April 2026. This role will form part of a new scrum team within Minerva Platform to develop and deliver the Ingestion and Risking within the SAS … implement automation test strategies and plans for SAS Viya 4 applications, for UI based Visual Analytics (VA) and Intelligent Decisioning (ID), as well as for integration testing of data pipelines. Develop and maintain test scripts using appropriate automation frameworks (e.g., Selenium, PyTest, JUnit) and integrate them into CI/CD pipelines. Confirm the test data strategy More ❯

table creation, SAS Studio V and SAS RTENG to build the SA registration network, SAS Viya 3.5 tools for risk assessment of new SA registrations. The POC leveraged data from 20 different sources, most of which were already housed in the Minerva Oracle database. Previously, some data had to be transferred manually. However, the automated file … transfers described in this Solution Design Document (SDD) will now move that data to the SAS platform using approved Enterprise Architecture (EA) integration patterns, with the initial phase, targeted for delivery in April 2026. This role will form part of a new scrum team within Minerva Platform to develop and deliver the Ingestion and Risking within the SAS … implement automation test strategies and plans for SAS Viya 4 applications, for UI based Visual Analytics (VA) and Intelligent Decisioning (ID), as well as for integration testing of data pipelines. Develop and maintain test scripts using appropriate automation frameworks (e.g., Selenium, PyTest, JUnit) and integrate them into CI/CD pipelines. Confirm the test data strategy More ❯

Batch Test Specialist + 6 month initial contract + £470 to £483 per day - Inside IR35 + Hybrid working form Telford Key Sklills: + Test automation for data and analytics platforms + SC Clearable Key Responsibilities: Working as part of an Agile team participate in ceremonies and contribute to sprint planning, retrospectives, demos and reporting. Utilise strong Stakeholder … implement automation test strategies and plans for SAS Viya 4 applications, for UI based Visual Analytics (VA) and Intelligent Decisioning (ID), as well as for integration testing of data pipelines. Develop and maintain test scripts using appropriate automation frameworks (e.g., Selenium, PyTest, JUnit) and integrate them into CI/CD pipelines. Confirm the test data strategy … CI/CD pipelines (e.g., Jenkins, GitLab CI). Troubleshoot and resolve issues in test environments, including SAS Viya containerized deployments (e.g., OpenShift, Kubernetes). Contribute to test data management and environment provisioning strategies Ensure that appropriate defect management is undertaken Ensure results and progress are clearly articulated to relevant stakeholders. Essential Skills & Experience: Proven experience in: Working More ❯

Batch Test Specialist + 6 month initial contract + 470 to 483 per day - Inside IR35 + Hybrid working form Telford Key Sklills: + Test automation for data and analytics platforms + SC Clearable Key Responsibilities: Working as part of an Agile team participate in ceremonies and contribute to sprint planning, retrospectives, demos and reporting. Utilise strong Stakeholder … implement automation test strategies and plans for SAS Viya 4 applications, for UI based Visual Analytics (VA) and Intelligent Decisioning (ID), as well as for integration testing of data pipelines. Develop and maintain test scripts using appropriate automation frameworks (e.g., Selenium, PyTest, JUnit) and integrate them into CI/CD pipelines. Confirm the test data strategy … CI/CD pipelines (e.g., Jenkins, GitLab CI). Troubleshoot and resolve issues in test environments, including SAS Viya containerized deployments (e.g., OpenShift, Kubernetes). Contribute to test data management and environment provisioning strategies Ensure that appropriate defect management is undertaken Ensure results and progress are clearly articulated to relevant stakeholders. Essential Skills & Experience: Proven experience in: Working More ❯

experienced Java Developer to join their growing technology team on a permanent basis. This role offers the opportunity to work on cutting-edge enterprise systems that power critical data-driven processes across the business. You’ll play a key role in developing and maintaining high-performance applications using Java 17, Spring Batch, Spring Integration, and GridGain Database, while … contributing to complex data workflow and integration projects. Key Responsibilities Design, develop, and maintain scalable Java 17 applications supporting enterprise data operations. Build, optimise, and manage batch processing solutions using Spring Batch. Design and implement integration flows with Spring Integration. Develop efficient SQL queries, stored procedures, and scripts to support data pipelines. Work with … GridGain Database to deliver distributed, high-performance data solutions. Support the transition and optimisation of data workflows from MongoDB to GridGain. Automate deployment and operational processes within Unix/Linux environments. Collaborate closely with cross-functional teams to deliver robust, secure, and scalable solutions. Skills & Experience Required Strong proficiency in Java 17 and familiarity with the More ❯

experienced Java Developer to join their growing technology team on a permanent basis. This role offers the opportunity to work on cutting-edge enterprise systems that power critical data-driven processes across the business. You’ll play a key role in developing and maintaining high-performance applications using Java 17, Spring Batch, Spring Integration, and GridGain Database, while … contributing to complex data workflow and integration projects. Key Responsibilities Design, develop, and maintain scalable Java 17 applications supporting enterprise data operations. Build, optimise, and manage batch processing solutions using Spring Batch. Design and implement integration flows with Spring Integration. Develop efficient SQL queries, stored procedures, and scripts to support data pipelines. Work with … GridGain Database to deliver distributed, high-performance data solutions. Support the transition and optimisation of data workflows from MongoDB to GridGain. Automate deployment and operational processes within Unix/Linux environments. Collaborate closely with cross-functional teams to deliver robust, secure, and scalable solutions. Skills & Experience Required Strong proficiency in Java 17 and familiarity with the More ❯