1 to 25 of 110 Data Pipeline Jobs in the North of England

Manchester (Data Engineer, Senior Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Data Engineer … proficiency, including experience with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience ...

with engineering best practices and the ability to translate customer expectations into applied technical functionality are key to success in this role. Key responsibilitiesData pipeline development Build and maintain scalable data pipelines using Apache Spark and PySpark, processing and transforming large datasets across distributed cloud infrastructure. Workflow … monitoring, and execution of data processing workflows. Root cause analysis Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues - including reviewing EMR output logs and CloudWatch metrics. Data modelling Apply understanding of dimensional data ...

small team with a large remit, and we expect AI-assisted development to be embedded in how we work Apply AI tools to accelerate pipeline development, automated testing, anomaly detection and data quality monitoring Stay current with AI developments in the data engineering space … this is non‐negotiable; you must have built and operated production data pipelines on this stack Strong Python skills – used daily for pipeline development, data transformation and automation Experience across at least two of the three core data domains: Customer, Commercial and Operations ...

Data Engineer We are looking for an experienced Data Engineer to join a major data transformation programme on an initial 3-month contract. This is a hands-on role for someone who enjoys building robust, scalable data platforms and pipelines. You will … work as part of a wider technical delivery team, helping to develop a modern AWS-based data environment that supports enterprise reporting, analytics and operational decision-making. The successful candidate will ideally be able to attend the Leeds office one day per week. However, flexible arrangements ...

automation in data engineering — this is a small team that needs to move fast, and we expect the use of AI‐assisted pipeline development and management to be embedded in how we work Data Governance Lead the Data Governance function from a standing … alongside a technology partner or platform team (similar to our Ingenuity relationship) Demonstrated use of AI in data engineering contexts — AI‐assisted pipeline development, automated quality monitoring, or similar Strong leadership and stakeholder management skills — able to influence at Director and C‐suite level Experience ...

Senior Data/AI Engineer - Manager £60,000 to 77,000 GBP Bonus Hybrid WORKING Location: Manchester, North West - United Kingdom Type: Permanent Senior Data/AI Engineer - Manager Location: Manchester Salary: Up to £77,000 + Bonus Employment Type: Permanent Security Clearance: Must be eligible … performance and solution reliability. Optimise data processing, storage and AI workloads for performance, scalability and efficiency. Produce clear technical documentation covering pipelines, data models, solution logic and technical assumptions. Ensure solutions adhere to appropriate data governance, security and privacy standards. Work as part ...

Senior Data Engineer - Remote We are looking for a Senior Data Engineer to take a key role in developing, modernising and improving the organisations data platform and data warehouse capability. This is a hands-on senior engineering position with genuine ownership. Youll … developing data warehouses, analytical platforms and data pipelines. Strong understanding of ETL/ELT, data transformation and pipeline architecture. Solid knowledge of data modelling, including dimensional modelling, Kimball, star schemas and/or medallion architecture. Hands-on experience with Microsoft Fabric ...

with our head office based in Liverpool city centre this is an exciting opportunity to come and join us as our Head Of Data as we embark on a data transformation journey. This is a hands on role supporting the migration of our legacy data across to our new cloud platform. This role would be ideal for a Head of Data, or a Lead Data & Analytics Engineer looking for that next step in their career. Client Details We are a 4th generation family owned business that was established ...

designs, develops, and operates critical digital environments that drive performance, resilience and sovereignty, helping public and private organizations worldwide retain control over their data and infrastructures, while meeting regulatory requirements. With more than 54,000 employees serving over 4,500 clients across 54 countries, Atos helps modernize core … with delivery teams. Support the development of High-Level Designs (HLDs), Low-Level Designs (LLDs), technical documentation, and knowledge articles. Assure data pipeline integrity throughout migration, testing, validation, and dual-running activities. Mentor and guide Informatica engineers and project teams throughout the delivery lifecycle. Skills ...

Cloud Data Engineer (Python) Up to £54,150 per annum + excellent pension | Hybrid working | London, Manchester, Cardiff, Belfast, Edinburgh or Darlington Are you an experienced Data or Software Engineer with hands-on experience of: Python and end to end Cloud deployment, looking for a role … where your work can make a real difference? Our Government client is looking for a Data Engineer to join its Data, Technology and Insights (DTI) unit on a 12-month fixed-term contract . Youll join a multidisciplinary team of data engineers, data ...

Senior Quantexa Data Engineer - Manchester/Hybrid (Quantexa Data Engineer, Lead Data Engineer, Senior Data Engineer, Data Engineering Manager, Quantexa, Spark, Scala, Python Databricks, DevOps, CI/CD pipelines, Jenkins, GitHub Actions, Azure DevOps, Docker, Kubernetes, Quantexa Data … global innovator and world leader with one of the most recognisable names within technology. They are looking for an experienced Quantexa Data Engineer with significant experience leading the technical development of Quantexa-based solutions. You will be tasked with designing and implementing robust data pipelines, configuring ...

Senior Quantexa Data Engineer - Leeds/Hybrid (Quantexa Data Engineer, Lead Data Engineer, Senior Data Engineer, Data Engineering Manager, Quantexa, Spark, Scala, Python Databricks, DevOps, CI/CD pipelines, Jenkins, GitHub Actions, Azure DevOps, Docker, Kubernetes, Quantexa Data … global innovator and world leader with one of the most recognisable names within technology. They are looking for an experienced Quantexa Data Engineer with significant experience leading the technical development of Quantexa-based solutions. You will be tasked with designing and implementing robust data pipelines, configuring ...

Permanent Opportunity available with a leading National Security Organisation for an eDV cleared Data Engineer in Manchester Manchester location – hybrid working when possible Must hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing … Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that help keep the nation safe, secure, and prosperous. You’ll work with cutting-edge technologies including AI/Data ...

Data Engineer Up to £55,000 We are working with a growing professional services organisation that is continuing to invest heavily in its data and technology capabilities. With data playing a central role in decision-making across the business, there is a strong focus … developing a modern, scalable data platform that supports reporting, analytics and operational performance. As part of this investment, they are looking for a Data Engineer to join their expanding Data team. This is an excellent opportunity for someone who enjoys building robust data ...

Data Engineer - Enterprise Knowledge & AI Platform 1 Vacancy | Leeds, Manchester or Edinburgh | Hybrid - 2 Days on Site Contract: Initial term to 20 December 2026, with a view to extend | multi-year programme We're looking for a Data Engineer to join a major UK financial services … exciting technology transformation. You'll be part of a multi-year programme building a new enterprise knowledge platform from scratch, transforming data and knowledge from across the organisation into trusted, searchable and AI-ready assets. This is a genuine greenfield opportunity where you'll help build the data ...

Lead Data Engineer - Up to 85K + Hybrid (2 days in office), 15% Pension, Private Healthcare + Microsoft Certifications - Leeds Central We are looking for a highly motivated and experienced Lead Data Engineer to join a growing digital and data function based in central … Leeds. This exciting opportunity will suit a talented data professional with strong experience across Microsoft Fabric, Azure and modern data platform technologies. You will be joining an established data function of 18 professionals supporting a £650 million revenue organisation with over 2,500 employees. ...

client, a leading Technology Consulting business based in Newcastle upon Tyne, seek a Data Engineer to join their elite Tyneside team as part of rapid expansion of the business to cope with client demand and continued excellent performance. About the Role As a Data Engineer … well as development, testing and maintenance of said platform to ensure the availability, reliability, and integrity of critical data. Key Deliverables Data Pipeline Development: Design and develop sophisticated data pipelines to import data from external vendors, sell-side partners, and other providers seamlessly ...

decide faster, and act with confidence across complex domains. We deliver situational awareness and information advantage across C5ISR, fusing cyber, geospatial, and kinetic data into an operational picture. This transforms fragmented data into a shared understanding of the battlespace, delivering actionable insight when it matters most. … secure-by-design, built from the ground up to protect sensitive information and ensure resilience in contested environments. We simplify complex heterogeneous data sources into a rich picture of fused intelligence, providing a digital backbone that underpins success. We operate as small teams delivering fast value, blending deep ...

Data Engineer £40,000 - 55,000 GBP Bonus, Pension, Benefits Hybrid WORKING Location: Newcastle Upon Tyne, North East - United Kingdom Type: Permanent Data Engineer Data & Cloud Engineering Location: Newcastle/Hybrid Working Salary: Up to £55,000 + Benefits Package About the Role … working with a leading technology and consulting organisation seeking a Data Engineer to join a growing engineering team delivering large-scale data solutions across a variety of complex projects. This role offers the opportunity to work with modern cloud technologies, real-time data platforms ...

Senior Data Engineer £50,000 - 68,000 GBP Bonus, Pension, Benefits Hybrid WORKING Location: Newcastle Upon Tyne, North East - United Kingdom Type: Permanent Senior Data Engineer Data & Cloud Engineering Location: Newcastle/Hybrid Working Salary: Up to £68,000 + Benefits Package About … Role We're supporting a major technology consulting business as they continue to expand their data engineering capability. As a Senior Data Engineer, you'll take a leading role in the delivery of complex cloud-based data platforms, helping organisations unlock greater value from ...

ready to shape the future of data? Matillion is the company behind Maia, the AI Data Automation platform. In the AI economy, demand for data is exploding. Manual data engineering can’t keep up. Maia fixes that by combining 15 years … Server) and NoSQL environments (MongoDB, Cassandra, DynamoDB). Hands-on command of integration architecture, including APIs, Linux environments, SQL, and data pipeline technologies. Active experience leading/deploying AI initiatives, specifically enterprise adoption of Generative and Agentic AI workflows (e.g., Claude, ChatGPT, Cursor, n8n). Preferred/ ...

platform infrastructure on Azure for a public sector client. You'll work with a reasonable degree of autonomy, contributing to infrastructure automation, pipeline reliability, and platform improvements, while collaborating closely with senior engineers, data engineers, and the wider delivery team. Key Responsibilities Support the build … platforms Work with Azure Data Factory, Azure Synapse Analytics, and Azure Data Lake to support data pipeline infrastructure and platform reliability Develop and maintain infrastructure-as-code using Terraform Build and maintain CI/CD pipelines using GitLab, supporting automated build ...

production-ready ML and AI systems Coach and guide a small team of MLOps Engineers, Analytics Engineers and AI Engineers Collaborate with Data Science, Pricing, Data Engineering and Software Development teams Evolve Machine Learning and AI Engineering standards and frameworks Enhance data pipelines … Operations (MLOps) Microsoft Azure Python Programming CI/CD Pipelines Coaching and Team Leadership Hard Skills Machine Learning AI Systems Development Data Pipeline Engineering Model Testing and Validation API Development Data Software Development Lifecycle Model Versioning Automated Monitoring Data Transformation Model Deployment Soft ...

Cloud Data Engineer (Python) Up to £54,150 per annum + excellent pension | Hybrid working | London, Manchester, Cardiff, Belfast, Edinburgh or Darlington Are you an experienced Data or Software Engineer with hands-on experience of: Python and end to end Cloud deployment, looking for a role … where your work can make a real difference? You'll join a multidisciplinary team of data engineers, data scientists, technologists, behavioural scientists and digital forensics experts, working on data processing and analytical projects. This is a hands-on role where you'll be designing ...

Data Engineer On going contract programme Hybrid | Competitive Salary + Benefits Are you a skilled Data Engineer looking to make a real impact? We're seeking a proactive and technically strong individual to help shape and develop a data platform using Microsoft Fabric . … This role will provide dedicated data engineering capability with a primary focus on developing and optimising the use of Microsoft Fabric. The team has already begun establishing capability in Fabric alongside ongoing delivery. This role will build on that foundation that support both current delivery and future growth. ...