801 to 825 of 1,184 Data Pipeline Jobs in the UK excluding London

Python engineer who thrives at the intersection of software and intelligence - where scalable systems meet real-world AI? This Cambridge-based team is building products that make complex, data-driven decisions seamless. Their work bridges the gap between cutting-edge machine learning and production-ready software used by global customers across sustainability, healthcare, and advanced engineering. You’ll … and building robust backend systems in Python using frameworks such as FastAPI, Django, or Flask Developing RESTful APIs and microservices that integrate seamlessly with machine learning models and data pipelines Implementing data processing and automation workflows, connecting real-world systems to intelligent decision layers Working closely with AI/ML engineers to deploy and scale models … systems in Python with proven experience at Senior level Experience with cloud platforms (AWS, Azure, or GCP) and containerisation (Docker, Kubernetes) Exposure to ML model integration, APIs, or data-driven systems (you don’t need to be a Data Scientist) A problem-solving mindset and passion for building software that drives impact Why join: Salary between More ❯

Python engineer who thrives at the intersection of software and intelligence - where scalable systems meet real-world AI? This Cambridge-based team is building products that make complex, data-driven decisions seamless. Their work bridges the gap between cutting-edge machine learning and production-ready software used by global customers across sustainability, healthcare, and advanced engineering. You’ll … and building robust backend systems in Python using frameworks such as FastAPI, Django, or Flask Developing RESTful APIs and microservices that integrate seamlessly with machine learning models and data pipelines Implementing data processing and automation workflows, connecting real-world systems to intelligent decision layers Working closely with AI/ML engineers to deploy and scale models … systems in Python with proven experience at Senior level Experience with cloud platforms (AWS, Azure, or GCP) and containerisation (Docker, Kubernetes) Exposure to ML model integration, APIs, or data-driven systems (you don’t need to be a Data Scientist) A problem-solving mindset and passion for building software that drives impact Why join: Salary between More ❯

enterprise-scale systems. Role & Responsibilities: Lead the design and delivery of enterprise applications and reporting systems Develop tools for dashboards, analytics, and operational monitoring Build and maintain APIs, data pipelines, and automation for business processes Implement scalable solutions to improve operational efficiency and data quality Enhance and integrate commercial off-the-shelf (COTS) platforms where required … automation, and Agile delivery Knowledge of compliance, regulatory reporting, and enterprise operational processes Proven experience delivering complex software solutions Strong leadership and stakeholder management skills Bonus: Python, Azure Data Factory/Databricks, data analysis, and COTS platforms Package: £100,000 15% non-contributory pension Hybrid working (3 days per week) Performance-based bonus scheme Comprehensive benefits More ❯

enterprise-scale systems. Role & Responsibilities: Lead the design and delivery of enterprise applications and reporting systems Develop tools for dashboards, analytics, and operational monitoring Build and maintain APIs, data pipelines, and automation for business processes Implement scalable solutions to improve operational efficiency and data quality Enhance and integrate commercial off-the-shelf (COTS) platforms where required … automation, and Agile delivery Knowledge of compliance, regulatory reporting, and enterprise operational processes Proven experience delivering complex software solutions Strong leadership and stakeholder management skills Bonus: Python, Azure Data Factory/Databricks, data analysis, and COTS platforms Package: £100,000 15% non-contributory pension Hybrid working (3 days per week) Performance-based bonus scheme Comprehensive benefits More ❯

enterprise-scale systems. Role & Responsibilities: Lead the design and delivery of enterprise applications and reporting systems Develop tools for dashboards, analytics, and operational monitoring Build and maintain APIs, data pipelines, and automation for business processes Implement scalable solutions to improve operational efficiency and data quality Enhance and integrate commercial off-the-shelf (COTS) platforms where required … automation, and Agile delivery Knowledge of compliance, regulatory reporting, and enterprise operational processes Proven experience delivering complex software solutions Strong leadership and stakeholder management skills Bonus: Python, Azure Data Factory/Databricks, data analysis, and COTS platforms Package: £100,000 15% non-contributory pension Hybrid working (3 days per week) Performance-based bonus scheme Comprehensive benefits More ❯

DATA ENGINEER - PRIVATE HEALTHCARE - SNOWFLAKE, AZURE, ETL, SQL POWER BI, SSRS, PYTHON Progressive, forward thinking, innovative international private healthcare organisation urgently require a talented Data Engineer to help then shape the future state of the organisation You will: Help create and develop data as they move forward into their new Snowflake environment to ensure … they deliver critical data in an accurate and timely information to the rest of the organisation. Develop and implement data pipelines that extract, transform and load data using into our Snowflake environment for use with reporting tools such as Power BI and SSRS. You are: An Experienced Data Engineer - ETL With in More ❯

Role: Azure Senior Data Engineer Location: Manchester, Lancashire Hybrid working: go into office 1x day a week Salary: £60,000 - £70,000 (dependent on experience) Insight into the Company: A large retail organisation - market leaders in their industry – are looking for a Senior Data Engineer to enter their team. You will be working in a small … team of 3 (including you!) and they are looking to build up the team over the next year. The ideal candidate will have experience with managing data warehouses from start to finish and liaise with IT teams regularly. The role will involve stakeholder management, working with diverse roles, such as software engineers, sales and marketing professionals and more. … Therefore, communicating your projects needs and understanding others is an essential part of this role. Role and Responsibilities: You will design, build and upgrade data pipelines You will work in CI/CD and with Software/DevOps teams in the organisation You will have expertise in azure – from collecting, to transforming to loading! Skills and Experience: Essential More ❯

Role: Azure Senior Data Engineer Location: Manchester, Lancashire Hybrid working: go into office 1x day a week Salary: £60,000 - £70,000 (dependent on experience) Insight into the Company: A large retail organisation - market leaders in their industry – are looking for a Senior Data Engineer to enter their team. You will be working in a small … team of 3 (including you!) and they are looking to build up the team over the next year. The ideal candidate will have experience with managing data warehouses from start to finish and liaise with IT teams regularly. The role will involve stakeholder management, working with diverse roles, such as software engineers, sales and marketing professionals and more. … Therefore, communicating your projects needs and understanding others is an essential part of this role. Role and Responsibilities: You will design, build and upgrade data pipelines You will work in CI/CD and with Software/DevOps teams in the organisation You will have expertise in azure – from collecting, to transforming to loading! Skills and Experience: Essential More ❯

Role: Azure Senior Data Engineer Location: Manchester, Lancashire Hybrid working: go into office 1x day a week Salary: £60,000 - £70,000 (dependent on experience) Insight into the Company: A large retail organisation - market leaders in their industry – are looking for a Senior Data Engineer to enter their team. You will be working in a small … team of 3 (including you!) and they are looking to build up the team over the next year. The ideal candidate will have experience with managing data warehouses from start to finish and liaise with IT teams regularly. The role will involve stakeholder management, working with diverse roles, such as software engineers, sales and marketing professionals and more. … Therefore, communicating your projects needs and understanding others is an essential part of this role. Role and Responsibilities: You will design, build and upgrade data pipelines You will work in CI/CD and with Software/DevOps teams in the organisation You will have expertise in azure – from collecting, to transforming to loading! Skills and Experience: Essential More ❯

We are currently looking for a Systems Engineer to join the Data Cloud team in our Cardiff Bay office. You will need to have demonstrable experience in Python and AWS cloud services. Creditsafe Privately owned and independently minded, Creditsafe operates with the singular vision of powering business decisions. We do this by delivering valuable intelligence on customers, suppliers … with a revolutionary dream to make business information accessible to all. Twenty-five years later, weve not only realised this dream, changed the market for the better, made data intelligence accessible to all businesses big and small but most importantly, opened new avenues of data intelligence for businesses with machine learning, AI and connected data. From … risk management through to opportunity identification, our industry-leading solutions, power decisions for companies by turning their data into actionable insights that help them become stronger, grow faster and thrive. The Team This opportunity is to join a team of highly technically skilled engineers, who are enhancing Creditsafes data offerings with high throughput and scalability as More ❯

with an organisation that’s passionate about your learning journey and committed to your professional career and personal development. Lead Technology Integration Workstream Drive the financial planning and data integration workstream within a broader transformation or modernization program. Align integration strategy with enterprise financial systems, ensuring data accuracy and consistency across planning, reporting, and forecasting platforms. … maintain a detailed roadmap covering milestones, dependencies, and risk mitigation plans. Oversee day-to-day execution, ensuring timely delivery and adherence to scope, quality, and compliance standards. Drive Data Integration, Migration, and Validation Lead the design and execution of data migration, validation, and reconciliation processes to support accurate financial and operational reporting. Collaborate with data and technology teams to ensure robust, secure data pipelines between core systems (e.g., ERP, EPM, data warehouse). Cross-Functional Coordination Serve as the bridge between Finance, Operations, Technology, and Data teams to ensure alignment on priorities, requirements, and timelines. Communicate effectively with stakeholders at all levels, translating technical concepts into actionable More ❯

with an organisation that’s passionate about your learning journey and committed to your professional career and personal development. Lead Technology Integration Workstream Drive the financial planning and data integration workstream within a broader transformation or modernization program. Align integration strategy with enterprise financial systems, ensuring data accuracy and consistency across planning, reporting, and forecasting platforms. … maintain a detailed roadmap covering milestones, dependencies, and risk mitigation plans. Oversee day-to-day execution, ensuring timely delivery and adherence to scope, quality, and compliance standards. Drive Data Integration, Migration, and Validation Lead the design and execution of data migration, validation, and reconciliation processes to support accurate financial and operational reporting. Collaborate with data and technology teams to ensure robust, secure data pipelines between core systems (e.g., ERP, EPM, data warehouse). Cross-Functional Coordination Serve as the bridge between Finance, Operations, Technology, and Data teams to ensure alignment on priorities, requirements, and timelines. Communicate effectively with stakeholders at all levels, translating technical concepts into actionable More ❯

with an organisation that’s passionate about your learning journey and committed to your professional career and personal development. Lead Technology Integration Workstream Drive the financial planning and data integration workstream within a broader transformation or modernization program. Align integration strategy with enterprise financial systems, ensuring data accuracy and consistency across planning, reporting, and forecasting platforms. … maintain a detailed roadmap covering milestones, dependencies, and risk mitigation plans. Oversee day-to-day execution, ensuring timely delivery and adherence to scope, quality, and compliance standards. Drive Data Integration, Migration, and Validation Lead the design and execution of data migration, validation, and reconciliation processes to support accurate financial and operational reporting. Collaborate with data and technology teams to ensure robust, secure data pipelines between core systems (e.g., ERP, EPM, data warehouse). Cross-Functional Coordination Serve as the bridge between Finance, Operations, Technology, and Data teams to ensure alignment on priorities, requirements, and timelines. Communicate effectively with stakeholders at all levels, translating technical concepts into actionable More ❯

Engineer (AWS) Location: Glasgow Duration: 31/12/2026 Days on site: 2-3 Rate: £402/day on Umbrella Role Description: We are currently migrating our data pipelines from AWS to Databricks, and are seeking a Senior Databricks Engineer to lead and contribute to this transformation. This is a hands-on engineering role focused on designing … building, and optimizing scalable data solutions using the Databricks platform. Key Responsibilities: Lead the migration of existing AWS-based data pipelines to Databricks. Design and implement scalable data engineering solutions using Apache Spark on Databricks. Collaborate with cross-functional teams to understand data requirements and translate them into efficient pipelines. Optimize performance … and cost-efficiency of Databricks workloads. Develop and maintain CI/CD workflows for Databricks using GitLab or similar tools. Ensure data quality and reliability through robust unit testing and validation frameworks. Implement best practices for data governance, security, and access control within Databricks. Provide technical mentorship and guidance to junior engineers. Must-Have Skills: Strong More ❯

unlock the value of technology and build a more sustainable, more inclusive world. Your Role We are looking for a skilled Spark/Scala Developer to join our data engineering team. The ideal candidate will have hands-on experience in designing, developing, and maintaining large-scale data processing pipelines using Apache Spark and Scala. You will … work closely with data scientists, analysts, and engineers to build efficient data solutions and enable data-driven decision-making. Key Responsibilities: Develop, optimize, and maintain data pipelines and ETL processes using Apache Spark and Scala. Design scalable and robust data processing solutions for batch and real-time data. Collaborate with … cross-functional teams to gather requirements and translate them into technical specifications. Perform data ingestion, transformation, and cleansing from various structured and unstructured sources. Monitor and troubleshoot Spark jobs, ensuring high performance and reliability. Write clean, maintainable, and well-documented code. Participate in code reviews, design discussions, and agile ceremonies. Implement data quality and governance best More ❯

unlock the value of technology and build a more sustainable, more inclusive world. Your Role We are looking for a skilled Spark/Scala Developer to join our data engineering team. The ideal candidate will have hands-on experience in designing, developing, and maintaining large-scale data processing pipelines using Apache Spark and Scala. You will … work closely with data scientists, analysts, and engineers to build efficient data solutions and enable data-driven decision-making. Key Responsibilities: Develop, optimize, and maintain data pipelines and ETL processes using Apache Spark and Scala. Design scalable and robust data processing solutions for batch and real-time data. Collaborate with … cross-functional teams to gather requirements and translate them into technical specifications. Perform data ingestion, transformation, and cleansing from various structured and unstructured sources. Monitor and troubleshoot Spark jobs, ensuring high performance and reliability. Write clean, maintainable, and well-documented code. Participate in code reviews, design discussions, and agile ceremonies. Implement data quality and governance best More ❯

unlock the value of technology and build a more sustainable, more inclusive world. Your Role We are looking for a skilled Spark/Scala Developer to join our data engineering team. The ideal candidate will have hands-on experience in designing, developing, and maintaining large-scale data processing pipelines using Apache Spark and Scala. You will … work closely with data scientists, analysts, and engineers to build efficient data solutions and enable data-driven decision-making. Key Responsibilities: Develop, optimize, and maintain data pipelines and ETL processes using Apache Spark and Scala. Design scalable and robust data processing solutions for batch and real-time data. Collaborate with … cross-functional teams to gather requirements and translate them into technical specifications. Perform data ingestion, transformation, and cleansing from various structured and unstructured sources. Monitor and troubleshoot Spark jobs, ensuring high performance and reliability. Write clean, maintainable, and well-documented code. Participate in code reviews, design discussions, and agile ceremonies. Implement data quality and governance best More ❯

unlock the value of technology and build a more sustainable, more inclusive world. YOUR ROLE We are looking for a skilled Spark/Scala Developer to join our data engineering team. The ideal candidate will have hands-on experience in designing, developing, and maintaining large-scale data processing pipelines using Apache Spark and Scala. You will … work closely with data scientists, analysts, and engineers to build efficient data solutions and enable data-driven decision-making. YOUR PROFILE Develop, optimize, and maintain data pipelines and ETL processes using Apache Spark and Scala. Design scalable and robust data processing solutions for batch and real-time data. Collaborate with … cross-functional teams to gather requirements and translate them into technical specifications. Perform data ingestion, transformation, and cleansing from various structured and unstructured sources. Monitor and troubleshoot Spark jobs, ensuring high performance and reliability. Write clean, maintainable, and well-documented code. Participate in code reviews, design discussions, and agile ceremonies. Implement data quality and governance best More ❯

unlock the value of technology and build a more sustainable, more inclusive world. YOUR ROLE We are looking for a skilled Spark/Scala Developer to join our data engineering team. The ideal candidate will have hands-on experience in designing, developing, and maintaining large-scale data processing pipelines using Apache Spark and Scala. You will … work closely with data scientists, analysts, and engineers to build efficient data solutions and enable data-driven decision-making. YOUR PROFILE Develop, optimize, and maintain data pipelines and ETL processes using Apache Spark and Scala. Design scalable and robust data processing solutions for batch and real-time data. Collaborate with … cross-functional teams to gather requirements and translate them into technical specifications. Perform data ingestion, transformation, and cleansing from various structured and unstructured sources. Monitor and troubleshoot Spark jobs, ensuring high performance and reliability. Write clean, maintainable, and well-documented code. Participate in code reviews, design discussions, and agile ceremonies. Implement data quality and governance best More ❯

unlock the value of technology and build a more sustainable, more inclusive world. YOUR ROLE We are looking for a skilled Spark/Scala Developer to join our data engineering team. The ideal candidate will have hands-on experience in designing, developing, and maintaining large-scale data processing pipelines using Apache Spark and Scala. You will … work closely with data scientists, analysts, and engineers to build efficient data solutions and enable data-driven decision-making. YOUR PROFILE Develop, optimize, and maintain data pipelines and ETL processes using Apache Spark and Scala. Design scalable and robust data processing solutions for batch and real-time data. Collaborate with … cross-functional teams to gather requirements and translate them into technical specifications. Perform data ingestion, transformation, and cleansing from various structured and unstructured sources. Monitor and troubleshoot Spark jobs, ensuring high performance and reliability. Write clean, maintainable, and well-documented code. Participate in code reviews, design discussions, and agile ceremonies. Implement data quality and governance best More ❯

opportunity to take on a high-impact, hands-on leadership role within a scaling global SaaS company that’s using AI to reinvent how internal business systems and data work together. As a Staff Software Engineer (AI/ML) , you’ll be the technical lead for a newly formed AI Engineering function , designing and delivering the internal AI … and data infrastructure that supports smarter decision-making across the organisation. You’ll build and scale AI-powered backend systems that connect enterprise data platforms (CRM, analytics, and communication tools) with large language models — creating intelligent tools and insights that enhance the way teams operate. This is a role with real influence: you’ll own the … adopted across the company. What You’ll Do Lead the design, architecture, and implementation of internal AI/ML systems for business intelligence. Build robust APIs, microservices, and data pipelines to power intelligent, data-driven tools. Develop retrieval-augmented generation (RAG) systems using vector databases for contextual AI. Set the technical direction for backend and AI More ❯

opportunity to take on a high-impact, hands-on leadership role within a scaling global SaaS company that’s using AI to reinvent how internal business systems and data work together. As a Staff Software Engineer (AI/ML) , you’ll be the technical lead for a newly formed AI Engineering function , designing and delivering the internal AI … and data infrastructure that supports smarter decision-making across the organisation. You’ll build and scale AI-powered backend systems that connect enterprise data platforms (CRM, analytics, and communication tools) with large language models — creating intelligent tools and insights that enhance the way teams operate. This is a role with real influence: you’ll own the … adopted across the company. What You’ll Do Lead the design, architecture, and implementation of internal AI/ML systems for business intelligence. Build robust APIs, microservices, and data pipelines to power intelligent, data-driven tools. Develop retrieval-augmented generation (RAG) systems using vector databases for contextual AI. Set the technical direction for backend and AI More ❯

opportunity to take on a high-impact, hands-on leadership role within a scaling global SaaS company that’s using AI to reinvent how internal business systems and data work together. As a Staff Software Engineer (AI/ML) , you’ll be the technical lead for a newly formed AI Engineering function , designing and delivering the internal AI … and data infrastructure that supports smarter decision-making across the organisation. You’ll build and scale AI-powered backend systems that connect enterprise data platforms (CRM, analytics, and communication tools) with large language models — creating intelligent tools and insights that enhance the way teams operate. This is a role with real influence: you’ll own the … adopted across the company. What You’ll Do Lead the design, architecture, and implementation of internal AI/ML systems for business intelligence. Build robust APIs, microservices, and data pipelines to power intelligent, data-driven tools. Develop retrieval-augmented generation (RAG) systems using vector databases for contextual AI. Set the technical direction for backend and AI More ❯

world’s leading organizations unlock the value of technology and build a more sustainable, more inclusive world. YOUR ROLE We are seeking a skilled and hands-on AWS Data Engineer with strong coding expertise and deep experience in building scalable data solutions using AWS services. The ideal candidate will have a solid background in data engineering, Python development, and cloud-native architecture. YOUR PROFILE Design, develop, and maintain robust data pipelines and ETL workflows using AWS services. Implement scalable data processing solutions using PySpark and AWS Glue. Build and manage infrastructure as code using CloudFormation. Develop and deploy serverless applications using AWS Lambda, Step Functions, and S3. Perform data querying and analysis using Athena. Collaborate with Data Scientists to operationalize models using SageMaker. Ensure secure and compliant data handling using IAM, KMS, and VPC configurations. Containerize applications using ECS for scalable deployment. Write clean, testable code in Python, with a strong emphasis on unit testing. Use GitLab for version control, CI/CD More ❯

achieve this through positivity, action, professionalism, customer service and industry knowledge. What you will do to be successful in this role as Technical Consultant Key Responsibilities Provide all data related activity of Technical Consultancy Services across several Products to Civica customers Develop and deliver new approaches to Technical delivery to continuously enhance the level of service provided. To … deliver data migration and data consultancy for Civica software following Civica project documentation, implementation guidelines and quality standards. Act as the lead point of contact for their Projects for the areas of Data, interfacing and Technical delivery. Define and document data conversion and Load scope and perform data loading in … reports via a range of industry standard reporting tools To promote Civica, its products and practices in a professional manner. Create SQL scripts to view and report on data Creation of SQL Stored Procedures, Functions and Triggers. Development of back office tools and data pipelines Migration of SSIS packages and legacy SQL approaches to Azure environments More ❯