25 of 25 Data Pipeline Jobs in Lancashire

Head of Data Engineering - Preston A Head of Data Engineering/Data Engineering Manager to lead the design, development, and enhancement of the data infrastructure and pipelines is required by a leading company based in Preston. The role offers Hybrid working, so 2 - 3 days in the office a week. You must … solutions for delivering high-quality care. The ideal candidate will lead the design and execution of cloud-based based scalable data storage solutions, oversee ETL pipeline development and optimisation, establish and manage data schemas and dictionaries, develop data integration solutions, and lead data cleansing, validation, and enrichment processes. Duties … You'll be on the lookout for performance bottlenecks and working to optimise data processing and query performance. This will help make sure your data pipeline is scalable and ready to handle big loads. You'll also be setting up monitoring frameworks to keep an eye on how your data pipeline is More ❯

we're proud to be consumer champions in the automotive space. We're constantly exploring smarter ways to connect people with the right cars - and that's where data engineering plays a key role. You'll join a focused data team of 4 professionals (2 Data Scientists, 1 BI Developer, and yourself as Data … CI/CD pipelines for our data infrastructure, ensuring automated testing and smooth deployments. Participating in code reviews and contributing to team standards for data pipeline development. Keeping on top of the latest datasets available within the automotive space and making recommendations about new data sources. Supporting and expanding our Microsoft Azure infrastructure … optimising it for data pipeline performance. Writing comprehensive data documentation for analytics-focused entities, accelerating your colleagues' understanding of available data. What You'll Need to Succeed Essential Requirements: 2-4 years of experience building robust data pipelines in a commercial environment or through complex personal projects. Strong Python skills including experience with More ❯

algorithms. Lead the selection, validation, and optimization of models to discover meaningful patterns and insights, ensuring models remain relevant, reliable, and scalable. Drive continuous integration and deployment of data science solutions, optimizing performance through advanced machine learning techniques, code reviews, and best practices. 'Develop and deliver sophisticated visualizations, dashboards, and reports translate complex data into clear … actionable insights for business stakeholders. Present technical solutions to business stakeholders, using creative methods to explain complex concepts, increase understanding, and encourage solution adoption. Mentor and develop junior data scientists, fostering a culture of continuous learning, knowledge sharing, and skills development within the organization. Write clean, high-quality code, ensuring all outputs pass quality assurance checks, and contribute … Experience with MLOps practices including model drift detection, decay, A/B testing, integration testing, differential testing, Python package building, and code version control. Skilled in data pipeline creation and working with both structured and unstructured data. Familiar with cloud platforms (AWS, Azure, GCP) and containerization technologies like Docker and Kubernetes. Excellent problem-solving skills, combined with More ❯

Remote Data Architect sought by a well-established company in their industry; our client is seeking an adept professional to spearhead strategic leadership and technical expertise within their data solutions landscape, focussed on cloud data platforms, but including IoT analytics, data integration, and migration projects. This unique opportunity is offered on a … remote-first basis, but with an open invitation to their headquarters any time. Role Responsibilities: The Data Architect will be pivotal in shaping the approach to designing and developing data solutions using cloud technologies such as AWS and Azure. This position involves: - Leading the design and implementation of innovative cloud data platforms. - Setting standards … using Azure, AWS, or exceptional on-premise design expertise. - At least 5 years in data engineering or business intelligence roles. - Proficiency in ETL and data pipeline design, with a technology-agnostic approach. - A solid understanding of data warehouse and data lake principles. - Expert SQL skills and demonstrable data modelling More ❯

Job Title: Data Product Manager - Fully Remote About the Company: Join a leading AI-driven company that helps enterprise brands and agencies optimise paid search and ads. Role Overview: We are looking for a technically strong, hands-on Data Product Manager to drive the evolution of our data platform and machine learning ecosystem. This … is a platform-focused role, deeply technical but not architectural, working across high-volume data pipelines and predictive ML models to improve data quality, scale, and performance. Key Responsibilities: Own the data product roadmap and define the vision for the data platform. Manage internal data projects with high customer impact. … Ensure integrity, quality, and performance of data models and pipelines. Collaborate closely with Data Engineering, Data Science and PMs Contribute to AI and GenAI initiatives in search intelligence. Key Requirements: 5-10 years of experience in data roles, with at least 3 years in Data Engineering or Data More ❯

Test Engineer Data Platforms (FTC-6months Hybrid | London or Manchester) My client is looking for an experienced Test Engineer to take ownership of testing across modern data platforms. This is a hands-on role where you’ll design, implement, and execute test strategies for data pipelines, models, and reporting solutions built on Azure technologies. … Key Responsibilities Define and own the overall testing strategy for data pipelines, data models, and BI solutions. Develop and implement automated testing frameworks integrated with CI/CD pipelines. Design and execute test cases across data ingestion, transformation, storage, and presentation layers. Validate data accuracy, performance, and usability within Azure Databricks, Azure … Data Factory, Azure Data Lake Storage, and Power BI. Establish and monitor data quality checks, reconciliation, and compliance processes. Collaborate closely with data engineers, BI developers, architects, and product owners to embed testing best practices across delivery. Essential Skills & Experience Proven experience as a Test Engineer within data engineering or More ❯

DV Cleared Data Engineers – Manchester – AWS or Azure – Senior/Principal We’re supporting a leading global consultancy with the expansion of their secure data engineering team, and we’re looking for DV Cleared Data Engineers to join major projects in Manchester. If you hold active DV clearance and want to work on cutting … edge, mission-critical programmes in a highly collaborative technical environment, this is an excellent opportunity. They are considering Data Engineers at all levels from Junior up to Principal Data Engineer. What You’ll Be Doing Designing and delivering secure, high-performance data pipelines and analytics platforms Working across AWS or Azure environments, building scalable … national infrastructure and security programmes with real impact What We’re Looking For Active DV clearance (essential) Strong background in data engineering, ETL, and data pipeline development Experience with AWS or Azure cloud technologies Excellent coding skills in Python, SQL, or Scala Understanding of data warehousing, modelling, and streaming technologies Confident working in More ❯

Data Architect Location: Manchester (Hybrid - Onsite Tuesdays) Salary: £85,000 - £90,000 + up to 25% Bonus A leading household name in the digital and e-commerce space, with over 16 million users on their app and £1 billion in annual revenue generated through digital channels, is embarking on a major 5-year data transformation strategy. … company-wide on-prem to Azure migration. What You'll Be Doing Lead the cloud architecture transformation (Azure focus) Work hands-on with SQL, Azure, and data pipeline development Engage with stakeholders across the business - educate, influence, and lead Collaborate with BI, Data Engineering, and Digital Analytics teams Design and deliver scalable architecture for massive … data sets and high-volume user bases What You'll Bring Minimum 2 years' experience in architecture (cloud/data focused) Strong hands-on technical background (SQL, Azure essential) Experience in on-prem to cloud migration (Azure preferred) Ability to work autonomously, with strong stakeholder management Experience working with large data volumes and customer More ❯

Data Delivery and Integration Manager (Remote) - Full time permanent role - Retail industry - Remote/home-based - Salary up to £75,000 plus bonus, private healthcare and more! A large and growing retail business are looking to expand their IT function with the addition of a Data Integration Manager. This role will focus on end to end … data engineering and integration solutions for a company wide modernisation programme. Duties will include: - Managing the processes for data engineering and data integration in order to align to business needs - Leading a team across engineering and integration - Work to ensure the data infrastructure of the business effectively supports analytics, AL, machine learning … ML and reporting - Designing of data pipelines, ETL, ELT processes and architecture - Work with internal stakeholders and multiple 3rd parties to seamlessly integrate systems, platforms and applications To be considered suitable for this Data Engineering Manager role you will need to have experience across the following: - Team leadership and guidance - Data engineering and aligning More ❯

Senior Data Engineer in Databricks - Lead the Future of Data in Financial Services Location: Hybrid | 2-3 days/week on site Offices: Banbury, Milton Keynes, Bedford, Northampton, Manchester Salary: Up to £80,000 + Car Allowance + Bonus + Benefits Note: Visa sponsorship is not available for this role. Be the Technical Leader in a … Bold Data Transformation Our client is entering a pivotal phase in its data journey. With a new Head of Data already in place and a Data Architect joining shortly, the business is investing in a modern, scalable data platform to turn messy, complex client data into trusted, actionable … team. Finance sector experience is a bonus-but not essential. Interview Process Initial coffee chat to explore team fit and technical challenges. Technical interview covering Databricks, MDM, semantic layers, pipeline design, and governance. This is a rare opportunity to lead, build, and shape the future of data in a growing financial services business. Ready to make your More ❯

This role involves designing, developing, and maintaining robust data pipelines to support analytics within the industrial and manufacturing sector. The successful candidate will be heavily involved in supporting our cloud migration transition. Client Details Based in Manchester City Centre, we are a leader in our field in the UK and Ireland We are a testing, inspection, certification, and … We serve over 35,000 customers with services like electrical testing, asset management, non-destructive testing (NDT), and inspections for infrastructure, manufacturing, and healthcare facilities. Description The Successful Data Engineer will be responsible for but not limited to: Development and implement data and reporting solutions from our Dynamics, in-house and 3rd party sources, using the … latest Microsoft technologies: Azure Synapse Analytics & Azure Data Factory, Azure Data Lake, Azure SQL Database, Support older Microsoft Technologies whilst we are in transition: SSIS and SSRS. Supporting change and migration efforts. Work with other members of the team or directly with business users to understand and document business requirements, evaluate options, research and propose suitable More ❯

Data Engineer Bolton- Hybrid £45,000 - £55,000 (dependent on experience) We're looking for a Data Engineer experienced in Generative AI to join our growing international and cross functional client. In this role, you'll be responsible for evaluating, building, and maintaining high quality data sets that support their internal stakeholders across the … business. You'll play a key part in ensuring data pipelines are secure, reliable, and scalable. Data Engineer Duties: Design, develop, and maintain robust data pipelines and architectures. Evaluate, build, and optimise data sets for internal customers. Ensure data systems comply with governance and security standards. Collaborate with internal teams … to understand data needs and deliver effective solutions. Stay ahead of emerging data and AI technologies, contributing to our innovation roadmap. Support the integration of generative AI capabilities into existing systems. As a Data Engineer, you will have: Strong experience with SQL (e.g., MS SQL, Oracle). Knowledge of NoSQL technologies (e.g., MongoDB, InfluxDB More ❯

The Senior Data Engineer will play a pivotal role in developing and optimising data pipelines, ensuring the seamless flow of information across the organisation. The successful candidate will also be heavily involved in supporting our cloud migration project. Client Details Based in Manchester City Centre, we are a leader in our field in the UK and … serve over 35,000 customers with services like electrical testing, asset management, non-destructive testing (NDT), and inspections for infrastructure, manufacturing, and healthcare facilities. Description The Successful Senior Data Engineer will be responsible for but not limited to: · Development and implement data and reporting solutions from our Dynamics, in-house and 3rd party sources, using the … latest Microsoft technologies: Azure Synapse Analytics & Azure Data Factory, Azure Data Lake, Azure SQL Database Support older Microsoft Technologies whilst we are in transition: SSIS and SSRS. Supporting change and migration efforts. Work with other members of the team or directly with business users to understand and document business requirements, Undertake/support the monitoring of More ❯

The Senior Data/BI Engineer will play a pivotal role in developing and optimising data pipelines, ensuring the seamless flow of information across the organisation. The successful candidate will also be heavily involved in supporting our cloud migration project. Client Details Based in Manchester City Centre, we are a leader in our field in the … serve over 35,000 customers with services like electrical testing, asset management, non-destructive testing (NDT), and inspections for infrastructure, manufacturing, and healthcare facilities. Description The Successful Senior Data Engineer will be responsible for but not limited to: · Development and implement data and reporting solutions from our Dynamics, in-house and 3rd party sources, using the … latest Microsoft technologies: Azure Synapse Analytics & Azure Data Factory, Azure Data Lake, Azure SQL Database Support older Microsoft Technologies whilst we are in transition: SSIS and SSRS. Supporting change and migration efforts. Work with other members of the team or directly with business users to understand and document business requirements, Undertake/support the monitoring of More ❯

Junior Data Scientist - Fully remote (UK based only) – TimeSeries, NLP, Computer Vision We’re partnering with a fast-growing analytics and technology business operating in the media streaming sector to recruit a Full Stack Data Scientist. They provide data analytics tools for major entertainment brands that are household names around the globe. This role … offers a chance to build models, generate insights, and work directly with real-world product and customer data across multiple data types - text, images, and time-series (tabular) data. The Role You’ll join a small, collaborative data team in a Full stack Data Science role (ETL process through to building predictive … evaluating time-series forecasts Prototyping AI/ML models (NLP and computer vision exposure) Translating findings into clear visual reports and concise business narratives Contributing to simple, reliable data pipelines and automated quality checks About You Minimum 1 year of commercial Data Science experience Strong hands-on Python (pandas, NumPy, scikit-learn) and SQL Sound understanding More ❯

+ Benefits Robert Walters are delighted to be supporting a well-known business in Manchester on an exclusive basis. We're searching for a Solutions Architect with a Data background, that wants to be part of an exciting transformation journey. Responsibilities: Gaining in-depth knowledge of existing business legacy systems and mapping migration strategies to Azure cloud-native … solutions. Designing future systems with built-in security to comply with regulatory requirements and company standards. Assessing and prioritising business needs and new requirements within the data architecture strategy to support evolving reporting and analytics. Driving innovation by leading solution development and facilitating technical workshops to align on architecture and design. Defining and managing relationships between data sources while overseeing the impact of future or proposed changes. As a Solutions Architect you will have: 7+ years of experience migrating systems and data platforms from on-premises data centres to public cloud environments (Azure preferred but not required). Deep knowledge of cloud services for data storage, data More ❯

Senior Analytics Engineer Whitefield, Manchester- Hybrid Up to £60,000 (DOE) + benefits As the Senior Analytics Engineer, you'll work closely with the Head of Data and business stakeholders to design, build, and optimise data pipelines and analytics solutions that power strategic decision-making. You'll be hands-on with SQL Server, Alteryx, Power BI … ensuring the platform is robust, scalable, and future-ready. Senior Analytics Engineer responsibilities: Partner with business teams to understand requirements and deliver meaningful, actionable insights. Design and implement data pipelines from APIs and relational sources. Model data effectively using Kimball/Star Schema methodologies. Develop dashboards, reports, and automated integrations in Power BI. Support the onboarding … of data from newly acquired businesses. Contribute to data strategy, process improvement, and best practice development. Stay ahead of emerging tools, technologies, and AI trends to continually enhance the analytics function. Essential skills and experience: Advanced SQL proficiency (joins, CTEs, window functions). Strong Power BI skills, including semantic modelling, DAX, and report design. Experience building More ❯

to inform and influence the Group’s future technology direction. Key Responsibilities: Explore and evaluate emerging technologies to identify opportunities for innovation and business impact. Design and deliver a pipeline of proofs of concept (PoCs) and early-stage projects aligned with the Innovation strategy. Act as a technical SME for our Reference Architecture, defining the evolution and optimisation of … this platform Collaborate with business stakeholders, architects, data SMEs, and delivery teams to shape priorities and approaches. Build and test prototypes, ensuring they are scalable and production-ready where appropriate. Lead infrastructure and environment setup for innovation projects, including CI/CD pipelines and cloud environments. Support the development of data pipelines, processing, and storage for … programming skills in Python, C#, or similar languages. Experience with CI/CD pipelines, containerisation (Docker/Kubernetes) Familiarity with cloud platforms (Azure, AWS, or GCP) Understanding of data engineering principles, including ETL, data pipelines, and database management. Working knowledge of ML/AI concepts and technologies, including model deployment and monitoring. Strong problem-solving skills More ❯

and development approach, with a strong emphasis on team collaboration. This is a stimulating environment where systems must operate in real-time, requiring robust event-driven architectures, streaming data pipelines, and reactive programming. You'll tackle complex scalability challenges across distributed systems, ensuring speed and reliability under heavy user loads. Security and compliance are central to the platform More ❯

and development approach, with a strong emphasis on team collaboration. This is a stimulating environment where systems must operate in real-time, requiring robust event-driven architectures, streaming data pipelines, and reactive programming. You'll tackle complex scalability challenges across distributed systems, ensuring speed and reliability under heavy user loads. Security and compliance are central to the platform More ❯

and development approach, with a strong emphasis on team collaboration. This is a stimulating environment where systems must operate in real-time, requiring robust event-driven architectures, streaming data pipelines, and reactive programming. You'll tackle complex scalability challenges across distributed systems, ensuring speed and reliability under heavy user loads. Security and compliance are central to the platform More ❯

the DNA of the platform What We’re Looking For Proven track record designing or architecting distributed, high-volume systems Experience with event-driven architectures and high-throughput data pipelines (Kafka) Strong knowledge of at least one of: C#, .NET, Golang, or TypeScript , plus SQL/databases Ability to balance hands-on technical leadership with high-level solution More ❯

help shape the future of our engineering function. Who We're Seeking We're searching for an experienced technical leader to own and drive our back-end and data strategy. You'll lead teams responsible for designing, developing, and optimising cloud-based systems, data pipelines, and APIs - ensuring our solutions are robust, secure, and aligned with … scalable solutions, fostering a high-performance culture within the engineering team. What You'll Be Doing Lead the design, development, and implementation of .NET back-end systems and data workflows using Databricks . Oversee the build and optimisation of data pipelines within AWS . Drive engineering excellence , establishing and maintaining standards, metrics, and best practices. Mentor … What You'll Bring Proven expertise in back-end development using .NET/C# . Strong technical leadership and experience managing engineering teams. Hands-on experience with AWS data services and cloud-native architectures. Proficiency in Python and SQL Excellent stakeholder management skills, able to bridge technical and business perspectives. Knowledge of Azure and experience with Databricks or More ❯

My client is a UK sovereign AI company developing secure, mission-critical artificial intelligence and data platforms for defence, national security, and strategic industry partners. Our systems operate at the intersection of Real Time data, autonomous decision support, and cloud-native infrastructure, with a strong emphasis on security, integrity, and performance. They are scaling rapidly and … engineers) delivering secure, scalable, and high-availability AI systems. Own AWS-based infrastructure: Lambda, EC2, S3, CloudFormation/Terraform, CloudWatch, and VPC security. Oversee JSON-based API integrations, data pipelines, and serverless event-driven architectures. Implement continuous integration and deployment (CI/CD) pipelines for code, models, and infrastructure. Ensure DevSecOps best practices - patching, logging, monitoring, and compliance … with UK MoD or ISO27001 frameworks. Manage incident response, uptime SLAs, and operational dashboards. Collaborate with software engineers, AI researchers, and data scientists to optimise cloud performance. Maintain documentation and enforce secure coding and deployment standards. Support accreditation, audits, and readiness for UK sovereign hosting environments (IL2-IL5). Report directly to the Head of Engineering/CTO More ❯

cross-functional environment. Experience in line management or team leadership, with a track record of developing and supporting engineers. AWS?(Lambda, Glue, ECS, S3, etc.) Python and PySpark?(data pipelines, APIs, automation) TypeScript and React?(frontend development) Excellent communication and stakeholder management skills. Demonstrated expertise in technical design and architecture of distributed systems. Familiarity with fraud detection models … anomaly detection, and rule-based systems. Clear understanding of secure coding practices and data privacy principles. Exposure to CI/CD pipelines and infrastructure-as-code tools (e.g., Terraform, CloudFormation). In return, you will be rewarded with a highly competitive salary, annual bonus, performance bonus and an enviable benefits package. Hybrid working 3 days a week in More ❯