51 to 72 of 72 Data Pipeline Jobs in the North of England

Job Title: Data Engineer Salary: £40k - £60k + Excellent Benefits Package Location: Huddersfield (Hybrid) Key Skills: SQL, PowerBI, Airflow Summary A new role has opened up for a Data Engineering with SQL, BI, Cloud and Airflow DAG to join a media-focused business. The role has opened up as the company are heavily investing, and have … exciting plans for their data team. i This a permanent position being offered on a hybrid basis with 2 days onsite from their Huddersfield office. The Company You would be joining a highly-regarded business with over 20 years' experience in their field. Their mission is to provide their impressive client-base with the tools to run their … an employee you will have access to retail discounts, flexitime, health and wellbeing programs, and company pension schemes. The Role You will play a key role in transforming data sets to access insights to access the company, and their client base. Your responsibilities will cover: Develop data models to support company Business Intelligence Write and optimize More ❯

Head of Data Engineering Manchester (Hybrid) Permanent Full-time Salary: £100,000 - £120,000 (DOE) A major UK financial services organisation is seeking a visionary Head of Data Engineering to lead and scale its enterprise-wide data platforms. Youll drive the design and delivery of modern data solutions across cloud, analytics, and … core banking systems — enabling smarter decisions, customer innovation, and regulatory excellence. Key Responsibilities: Lead and develop high-performing data engineering teams Architect scalable, secure data pipelines and platforms (Azure, Databricks, etc.) Partner with product, risk, and tech leaders to deliver data-driven outcomes Champion best practices in data governance, quality, and DevOps … What You'll Bring: Proven leadership in large-scale data engineering within complex environments Deep technical expertise in cloud-native data architecture Strong strategic thinking and stakeholder engagement skills Passion for transforming legacy systems into modern, value-driving platforms This is a rare opportunity to shape the future of data in a trusted institution More ❯

DATA ENGINEER £65,000 + BENEFITS Remote (UK based) Looking for a role where you can take ownership of a modern cloud data warehouse, optimise pipelines, and make data more scalable, reliable, and cost-effective? THE COMPANY: I'm working with a global retail & e-commerce brand that's expanding rapidly across the UK. … With a growing digital and physical presence, their central data team plays a key role in powering insights across e-commerce, retail, finance, and marketing. THE ROLE: This is a hands-on data engineering role where you'll focus on building, optimising, and scaling pipelines into BigQuery (GCP) from a wide range of sources. You'll … work closely with BI developers and analysts to ensure clean, reliable data is available for reporting and decision-making. Manage and optimise pipelines from ERP, website, APIs, app stores, social media, and more. Work with BigQuery, Google Buckets, Python scripts, and Talend (transitioning to Dataform/Cloud workflows). Support migrations between GCP projects and ensure the warehouse More ❯

Role: Analytics and AI Manager The Team: A highly passionate bunch, our data, analytics and AI team bring our unique software solutions to life. Our mission is to empower our customers and internal stakeholders to improve business performance and minimise risk by providing them with clean, reliable data and analytical models. We are with them to … help protect their information, make informed decisions, deliver a personalised service and automate their workloads. Our engineers might be building data pipelines, modelling data, developing predictive models or automations for repeatable tasks or creation of data products. The role: Your role in the division is to lead the development of our MI, BI and … AI capabilities, exploiting our data warehouse to replace legacy reporting and create new BI and AI products for both internal and external customers. Your initial objectives will be to understand our business data and existing solutions and to define a roadmap for moving these into our Power BI cloud service. You will also work with other More ❯

passionate about continuous learning and delivering high-quality Python solutions. Key Responsibilities Design, develop, and test secure software systems using Python + React Build and maintain APIs and data pipelines using modern architecture principles Deploy solutions to cloud environments (AWS, Azure, and private cloud) Contribute across the full software development lifecycle from requirements gathering through to delivery and More ❯

Data Scientist (Hybrid | Manchester) A forward-thinking data company, known for delivering actionable insights across media and market research, is on the lookout for a Data Scientist. This role is ideal for someone who thrives in autonomy, enjoys working across teams, and wants to play a key part in building scalable, standardised systems that … ensure data reliability across multiple products. ?? The Opportunity: Join a product and data-driven company that works with top global brands Lead the development of systems to monitor data integrity, health, and performance Own projects end-to-end from defining the problem to deploying production-ready solutions Hybrid role: Split your time between our … Manchester office (Trafford Park) and remote working, with flexibility built in ?? What You'll Be Doing: Design and implement scalable monitoring and alerting solutions across data pipelines Define critical metrics and thresholds in collaboration with technical and non-technical teams Build and deploy models using Python, PySpark, and AWS services Troubleshoot data anomalies and promote best More ❯

Skills/requirements Deploy comprehensive cloud infrastructure for various products, including Astronomer Airflow and AccelData environments. Facilitate cross-functional integration between vendor products and other systems, such as data lakes, storage, and compute services. Establish best practices for cloud security, scalability, and performance. Manage and configure vendor product deployments, ensuring the setup and maintenance of environments. Ensure high … planning, resource allocation, and optimization of Airflow workers. Maintain and configure integrations with source control systems (e.g., GitHub, GitLab) for version control. Collaborate with cloud providers (e.g., AWS) for pipeline integration and scaling requirements. Identify, design, and implement internal process improvements, including re-designing infrastructure for greater scalability, optimizing data delivery, and automating manual processes. Develop infrastructure … using AWS and SQL technologies. Work with stakeholders, including design, product, and executive teams, to address platform-related technical issues. Build analytical tools to leverage the data pipeline, providing actionable insights into key business performance metrics, such as operational efficiency and customer acquisition.? All profiles will be reviewed against the required skills and experience. Due to the More ❯

Data Engineer - Award-Winning AI SaaS platform We're partnered with a pioneering SaaS company that's transforming their industry through cutting-edge AI and data technology. With a multi-award-winning platform and a client base that includes global household names, this is a rare opportunity to join a high-growth tech business that blends … startup agility with enterprise stability. We're looking for a passionate Data Engineer who is experienced with AI tooling - including building Gen AI agents in-product - to join their growing Data Engineering ream. Why You'll Love This Role: Tech at Scale : Work with hundreds of millions of data points daily, using distributed systems … office if you prefer Exceptional Benefits : From unlimited holiday and private healthcare to stock options and paid parental leave. What You'll Be Doing: Build and maintain scalable data pipelines using Spark with Scala and Java, and support tooling in Python Design low-latency APIs and asynchronous processes for high-volume data. Collaborate with Data Science More ❯

Data Engineer - Award-Winning AI SaaS platform We're partnered with a pioneering SaaS company that's transforming their industry through cutting-edge AI and data technology. With a multi-award-winning platform and a client base that includes global household names, this is a rare opportunity to join a high-growth tech business that blends … startup agility with enterprise stability. We're looking for a passionate Data Engineer who is experienced with AI tooling - including building Gen AI agents in-product - to join their growing Data Engineering ream. Why You'll Love This Role: Tech at Scale : Work with hundreds of millions of data points daily, using distributed systems … office if you prefer Exceptional Benefits : From unlimited holiday and private healthcare to stock options and paid parental leave. What You'll Be Doing: Build and maintain scalable data pipelines using Spark with Scala and Java, and support tooling in Python Design low-latency APIs and asynchronous processes for high-volume data. Collaborate with Data Science More ❯

smooth and efficient functioning and development of our databases and pipelines. You will be responsible for developing, optimising and managing these databases and ensuring a high-level of data quality and integrity. You will be defining the principles and processes that underpin data management while ensuring we maintain the highest standards for data management … repeatable and scalable. Create and manage database schemas, tables, stored procedures, indexes, and triggers. Ensuring high availability of databases through clustering, replication, and other techniques. Implement and maintain data pipelines and ETL processes for real-time and batch data ingestion. Monitor database performance and proactively address issues related to tuning and optimization. Ensure data security, privacy, integrity, and compliance with internal and external regulations. Collaborate with third-party suppliers, software developers, data analysts, DevOps, and other stakeholders to meet project requirements. Plan and implement database backups, restores, and disaster recovery strategies. Taking an active role in supporting and in finding the root cause of operational issues and preventing recurrences. Support More ❯

smooth and efficient functioning and development of our databases and pipelines. You will be responsible for developing, optimising and managing these databases and ensuring a high-level of data quality and integrity. You will be defining the principles and processes that underpin data management while ensuring we maintain the highest standards for data management … repeatable and scalable. Create and manage database schemas, tables, stored procedures, indexes, and triggers. Ensuring high availability of databases through clustering, replication, and other techniques. Implement and maintain data pipelines and ETL processes for real-time and batch data ingestion. Monitor database performance and proactively address issues related to tuning and optimization. Ensure data security, privacy, integrity, and compliance with internal and external regulations. Collaborate with third-party suppliers, software developers, data analysts, DevOps, and other stakeholders to meet project requirements. Plan and implement database backups, restores, and disaster recovery strategies. Taking an active role in supporting and in finding the root cause of operational issues and preventing recurrences. Support More ❯

A fantastic opportunity has arisen for a person to join a department to assist with the delivery of innovative generative AI data solutions. This role performs the duties of a Data Engineer specialising in generative AI and reports to the senior lead within a developing international and transversal structure. Key Responsibilities: Evaluate, design, deploy, improve and … support data sets for internal customers, ensuring resilience, security and responsiveness. Collaborate with internal stakeholders to optimise and secure their data in line with operational needs. Maintain and enhance data pipelines, ensuring compliance with data governance standards. Provide expertise in data management and quality assurance. Stay up to date with … the industrial and/or defence sector. The successful candidate must have: SQL technologies skills (e.g. MS SQL, Oracle). noSQL technologies skills (e.g. MongoDB, InfluxDB, Neo4J). Data exchange and processing skills (e.g. ETL, ESB, API). Development skills (e.g. Python). Big data technologies knowledge (e.g. Hadoop stack). This position offers a lucrative More ❯

Data Engineer – Manchester (Hybrid) Up to £55K to £65,000 + Bonus, 10% pension, 25+ Days HolidayPyspark | Databricks | SQL Shape the future of travel with data. Join a tech-led company that’s reimagining how millions experience travel. This is your chance to work on high-impact data projects that drive smarter decisions across product, operations … and commercial teams — all while growing your skills in a modern, cloud-first environment. Why This Role Stands Out: You’ll build scalable, production-grade data pipelines using PySpark , SQL , and Databricks Your work will directly power analytics, BI, and data science across the business You’ll be part of a collaborative, forward-thinking data … values experimentation and clean engineering You’ll get space to grow, learn new tools, and make a real impact What You’ll Bring: 2–5 years’ experience in data engineering or similar Strong PySpark and advanced SQL skills Hands-on experience with Databricks and building ETL/ELT pipelines Familiarity with CI/CD and version control Bonus More ❯

on in code and c30% leading - making key architectural decisions, mentoring, and setting technical direction. Responsibilities Own the architecture of the software systems, from drone control to cloud data pipelines Build backend control systems in Go, with critical components in C++ Design scalable cloud orchestration strategies (Railway, AWS, Kubernetes) Deliver advanced features including multi-drone (M:N) coordination … growing team of developers across multiple disciplines Drive technical standards, code quality, and test-driven development Upcoming Projects: Multi-drone coordination and control systems Next-gen data pipeline and product architecture Distributed and cloud-native scaling of core systems Advanced weather-intelligence and autonomy features Our Tech Stack Backend: Go C++ (Either or both - will allow upskill More ❯

A fantastic opportunity has arisen for a person to join a department to assist with the delivery of innovative generative AI data solutions. ** Please note that due to the nature of the client’s business, only candidates who currently hold full sole British Citizenship (without limitations) will be considered. ** This role performs the duties of a Data Engineer specialising in generative AI and reports to the senior lead within a developing international and transversal structure. Key Responsibilities: Evaluate, design, deploy, improve and support data sets for internal customers, ensuring resilience, security and responsiveness. Collaborate with internal stakeholders to optimise and secure their data in line with operational needs. Maintain and enhance … data pipelines, ensuring compliance with data governance standards. Provide expertise in data management and quality assurance. Stay up to date with evolving technology, contribute to the technology roadmap, and deliver cutting-edge solutions. The successful candidate will ideally have: Knowledge in NLP (Natural Language Processing). Knowledge in OCR (Object Character Recognition). Knowledge More ❯

A great marketing digital client based in Old Trafford is looking for a Data Scientist to help build and mature our monitoring and alerting infrastructure, centralising current efforts and enabling scalable, standardised approaches across multiple projects. This is a pivotal role for someone who thrives in autonomous environments and enjoys owning solutions from concept to deployment. The role … training programs, and all the support you need to keep growing. · Giving Back: Join us in supporting local charities and making a positive impact. The Role: As a Data Scientist in this role, you will lead the design, development, and deployment of systems that monitor the health and integrity of our data. You will work with stakeholders across … a hands-on role that requires both analytical thinking and strong engineering practices. You will take full ownership of projects - from discovering and defining problems, cleaning and preparing data in the warehouse, developing and testing models and logic, and deploying solutions into production environments. Key Responsibilities: · Design and implement monitoring and alerting systems to ensure the reliability and More ❯

Role: Azure Senior Data Engineer Location: Manchester, Lancashire Hybrid working: go into office 1x day a week Salary: £60,000 - £70,000 (dependent on experience) Insight into the Company: A large retail organisation - market leaders in their industry - are looking for a Senior Data Engineer to enter their team. You will be working in a small … team of 3 (including you!) and they are looking to build up the team over the next year. The ideal candidate will have experience with managing data warehouses from start to finish and liaise with IT teams regularly. The role will involve stakeholder management, working with diverse roles, such as software engineers, sales and marketing professionals and more. … Therefore, communicating your projects needs and understanding others is an essential part of this role. Role and Responsibilities: You will design, build and upgrade data pipelines You will work in CI/CD and with Software/DevOps teams in the organisation You will have expertise in azure - from collecting, to transforming to loading! Skills and Experience: Essential More ❯

web applications, focusing on lead generation workflows. Optimize and scale platform performance to handle increasing lead volumes efficiently. Integrate and manage APIs for customer eligibility checks, ensuring encrypted data transmission and receipt in compliance with GDPR and FCA guidelines. Collaborate with marketing teams and partners to enhance lead qualification processes using automated systems. Build responsive, conversion-driven user … interfaces with a seamless customer journey. Ensure platform security, including data encryption and compliance with UK regulatory standards. Troubleshoot, debug, and upgrade existing systems to improve functionality and user experience. Work with cloud infrastructure (AWS or Replit) to deploy, manage, and scale applications. Participate in code reviews, testing, and documentation to maintain high-quality standards. Stay updated on … and Express.js for server-side logic. Familiarity with Formik and Yup for form handling and validation. Database and Storage : Experience with Supabase or similar PostgreSQL-based solutions for data management. Ability to integrate and manage relational databases securely. API Integration : Strong experience in working with RESTful APIs (e.g., Numverify, ZeroBounce) to send and receive encrypted data. Knowledge of More ❯

Manager, you will be responsible for the pricing of multiple motor and home products, including the following core activities: Maintaining and improving behavioural models, prioritising a range of data science techniques. Advance the adoption of data science & statistical techniques. Develop reporting structures to monitor pricing performance in an automated fashion. Develop innovative solutions to improve rating … increasingly competitive insurance market. Key Responsibilities: Maintain and improve the pricing process to meet business requirements, including; Development of market leading behavioural modelling using a combination of traditional & data science techniques Improve sophistication within the optimisation of Markerstudy’s direct brands Support delivery of pricing & underwriting business plan across Motor & Home portfolios Support development of new product offerings … Advance the adoption of the latest data science & statistical techniques across pricing & underwriting Research and leverage new and existing data sources; capturing and explaining trends with innovative data features Clearly communicate results and proposed actions to key decision makers across the business Champion and facilitate automation of reporting, data pipelines and quality More ❯

Manager, you will be responsible for the pricing of multiple motor and home products, including the following core activities: Maintaining and improving behavioural models, prioritising a range of data science techniques. Advance the adoption of data science & statistical techniques. Develop reporting structures to monitor pricing performance in an automated fashion. Develop innovative solutions to improve rating … increasingly competitive insurance market. Key Responsibilities: Maintain and improve the pricing process to meet business requirements, including; Development of market leading behavioural modelling using a combination of traditional & data science techniques Improve sophistication within the optimisation of Markerstudy's direct brands Support delivery of pricing & underwriting business plan across Motor & Home portfolios Support development of new product offerings … Advance the adoption of the latest data science & statistical techniques across pricing & underwriting Research and leverage new and existing data sources; capturing and explaining trends with innovative data features Clearly communicate results and proposed actions to key decision makers across the business Champion and facilitate automation of reporting, data pipelines and quality More ❯

The Opportunity Reporting to the Associate Director of Data & Systems Engineering, this senior role sits at the heart of a multi-year digital transformation programme. You'll take the lead in managing, modernising, and integrating our core finance and HR systems, working closely with our in-house Development and Data teams, alongside key stakeholders, and third … party vendors across our international operations. Key Responsibilities Own the technical operation and performance of key Finance and HR applications. Design and implement solutions, integrations, and data pipelines using the Microsoft stack. Collaborate with global stakeholders and internal teams to define requirements, drive transformation, and ensure system integrity. Provide technical leadership in systems architecture, integration, and future-state … strategy. Troubleshoot, optimise, and maintain legacy systems while managing technical debt. Work with infrastructure and data engineering teams to support associated databases and servers. Your Background We're looking for a strong technical leader with: Proven experience working with finance systems and stakeholders in large international organisations. Deep expertise in Microsoft technologies and system integrations. A hands-on More ❯

The Opportunity Reporting to the Associate Director of Data & Systems Engineering, this senior role sits at the heart of a multi-year digital transformation programme. You'll take the lead in managing, modernising, and integrating our core finance and HR systems, working closely with our in-house Development and Data teams, alongside key stakeholders, and third … party vendors across our international operations. Key Responsibilities Own the technical operation and performance of key Finance and HR applications. Design and implement solutions, integrations, and data pipelines using the Microsoft stack. Collaborate with global stakeholders and internal teams to define requirements, drive transformation, and ensure system integrity. Provide technical leadership in systems architecture, integration, and future-state … strategy. Troubleshoot, optimise, and maintain legacy systems while managing technical debt. Work with infrastructure and data engineering teams to support associated databases and servers. Your Background We're looking for a strong technical leader with: Proven experience working with finance systems and stakeholders in large international organisations. Deep expertise in Microsoft technologies and system integrations. A hands-on More ❯