1 to 25 of 33 Permanent Data Pipeline Jobs in Lancashire

allowance based on a skills assessment Published on 10 July 2025 Deadline 30 July 2025 Location Croydon, Manchester, Sheffield About the job Job summary As a Lead Data Architect, you will create and uphold data architecture across critical Home Office systems, enabling secure, data-driven services that keep the country safe. Working under the … vision set by the Principal Data Architect, you'll design robust data models, metadata systems, and governance frameworks to serve diverse stakeholders. Your leadership will span strategic planning, mentoring junior data architects, and aligning technical decisions with business objectives. If you thrive on architecting solutions that integrate emerging technologies such as cloud-native platforms … data lakehouses, streaming data pipelines and AI-powered analytics while collaborating with multi disciplinary teams, this role offers you the chance to elevate how the Home Office exploits data for maximum public impact. Due to business requirements this post is available on a full time/flexible working basis. Where business needs allow, some More ❯

As a Lead Data Architect, you will create and uphold data architecture across critical Home Office systems, enabling secure, data-driven services that keep the country safe. Working under the vision set by the Principal Data Architect, you'll design robust data models, metadata systems, and governance frameworks to serve … diverse stakeholders. Your leadership will span strategic planning, mentoring junior data architects, and aligning technical decisions with business objectives. If you thrive on architecting solutions that integrate emerging technologies such as cloud-native platforms, data lakehouses, streaming data pipelines and AI-powered analytics while collaborating with multi disciplinary teams, this role offers you the … chance to elevate how the Home Office exploits data for maximum public impact. In this role, you will translate strategic data vision into tangible designs and solutions, ensuring data assets are secure, well-structured, and aligned with evolving Home Office priorities. Partnering with Technical Architects, Principal Data Architects, and various data More ❯

DEPT/AI has a single mission: to make the best work in the industry using Data & AI to enhance everything we do. This role sits within our Data & AI practice, which has deep expertise in leveraging AI. The team includes data strategists, consultants, data scientists and analysts that work alongside DEPT … and most challenging problems facing some of the best loved brands in the world - and doing this alongside an experienced team. This role is part of our EMEA Data craft team, an integrated craft across Europe driving data engineering, data science, and AI initiatives. We combine our expertise in data architecture, analytics … and AI to deliver exceptional data-driven solutions that build lasting relationships with our clients like Footlocker, Nikon, Rituals, and Philips ROLE OVERVIEW The Lead Data Engineer is responsible for building and leading our data engineering capabilities across EMEA and setting technical standards. This senior technical leadership role will drive the strategic vision for More ❯

hunt for builders . No, we've not ventured into construction in our quest to conquer the world, rather a designer and builder of systems for all things data related where we are conquering the Data World. We can offer an interesting insight into projects spanning a variety of sectors, which may include industries such as … to explore our dynamic team where you can truly act as an expert in your field in support of our clients and their challenges in the world of data and technology, read on! The Team We're Intuita - a fast growing consultancy that's making waves in both the consultancy and technology space. With our ambitious goals for … in the data pipelines. • Stay updated with the latest Azure technologies and industry best practices to continuously enhance data engineering capabilities. Looking at our current pipeline of work, we can also consider those with an Analytics Engineering lean, with experience within BigQuery (GCP) with data modelling in DBT and mobile/telecoms industry More ❯

Location: Worcester Duration: 6 month initial contract Rate: (Outside IR35) Security: Active DV clearance required Role details: We are looking for 3 x Data Engineers to join our defence & security client on a contract basis. You will helpdesign, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi … These roles are supporting our clients team in Worcester (fully onsite), and requires active UK DV clearance. Key Responsibilities: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi Implement data ingestion, transformation, and integration processes, ensuring data quality and security Collaborate with data … cross-functional teams. Detail-oriented with a focus on ensuring data accuracy, quality, and security. Proactive problem-solving mindset and ability to troubleshoot complex data pipeline issues Desirable skills: Experience working in government, defence, or highly regulated industries with knowledge of relevant standards. Experience with additional data processing and ETL tools like Apache More ❯

Senior Machine Learning Operations Engineer Location: Guernsey, Haywards Heath, Home Office (Remote) or Manchester Salary: £65,000 - £75,000 - depending on experience Department: Technology and Data We're 1st Central, a market-leading insurance company utilising smart data and technology at pace. Rapid growth has been based on giving our 1.4 million customers exactly what they … our colleagues too; we won Insurance Employer of the Year at the British Insurance Awards 2024 and our Glassdoor score is pretty mega too! We're big on data: it gives us the insights we need to give the right cover to the right customers at the right price. But it's the people inside and outside our … Deploying Infrastructure as Code (IAC) across various environments such as dev, uat and prod Handling large volumes of data in various stages of the data pipeline, from ingestion to processing Proven experience with feature stores, using them for both offline model development and online production usage Building integrations between cloud-based systems using APIs, specifically More ❯

UK Head Office - Manchester Hybrid/PERMANENT/SALARY: COMPETITIVE Job Purpose The Data Lead is responsible for leading and growing MPM's data capabilities. Managing the data team, overseeing the design, delivery, and governance of data infrastructure, pipelines, and analytics solutions. The Data Lead is also responsible for setting … and executing the vision for data within the organisation, enabling accurate, timely, and actionable insights through data practices, collaboration, and strong leadership. This role deals with a wide variety of tasks including, but not limited to the following: Manage, mentor, and grow the newly formed Data Team to deliver high-quality solutions that meet … evolving business objectives. Define and maintain the roadmap for data infrastructure, tools, and quality standards. Lead a data-first culture across the organisation, championing data literacy and self-service capabilities. Represent the Data Team in leadership discussions and ensure alignment with business priorities. Continuously improve data operations, governance, and platform More ❯

Group Data Engineering Lead £100,000 to £110,000 plus great benefits, including up to 10% employer pension contributions, private healthcare, and a company car or car allowance (Work Level 4) Manchester City Centre (Hybrid) We're looking for a Group Data Engineering Lead to join our Digital Technology & Data team in Manchester to … help take our data driven decision-making to the next level. We already have modern cloud platforms in place (Databricks, Azure) and use data far and wide across Co-op. We want your help to scale our team, empower more self-service, and help our Data Engineers be even more impactful with their work. … in a hybrid way - spending 1 to 2 days per week working from the office (find out more about our hybrid working policy at ). As our Group Data Engineering Lead, you'll lead the development of data connections and pipelines through our Lakehouse architecture - feeding BI and Data Science as well as securely More ❯

Data Engineer - Leading Pharmaceutical Company - Manchester(Tech Stack: Data Engineer, Databricks, Python, Power BI, Azure, TSQL, ETL, Agile Methodologies)About the Role: We are seeking a talented and experienced Data Engineer on behalf of our client, a leading Software House. This is a fully remote position, offering the opportunity to work with cutting-edge … technologies and contribute to exciting projects in a collaborative environment.About the Role: Our client is seeking an experienced Data Engineer to join their team in Manchester. This hybrid position involves working within the pharmaceutical industry, focusing on the design, development, and maintenance of data pipelines, ETL processes, and databases. The role is ideal for someone passionate … about improving processes, ensuring data quality, and maintaining compliance with regulatory standards. focusing on designing, developing, and maintaining data pipelines, ETL processes, and databases. If you are passionate about driving continuous improvement and ensuring data quality and compliance, we want to hear from you.Key Responsibilities:Design, develop, maintain, and optimise data pipelines More ❯

Our Senior Data Engineers enable public sector organisations to embrace a data-driven approach by providing high-quality, cost-efficient data platforms and services tailored to clients' needs. They develop, operate, and maintain these services, ensuring maximum value for data consumers, including analysts, scientists, and business stakeholders. Key responsibilities As a Senior … Data Engineer, you may assume multiple roles based on our clients' needs. The role is highly hands-on, supporting project delivery as a senior contributor and upskilling client team members. You might also take on a technical architect role, collaborating with the MadeTech team to identify growth opportunities within the account. You'll need a drive to deliver … criteria. Enthusiasm for learning and self-development Proficiency in Git (including Github Actions) and understanding of branch strategies Experience gathering and meeting requirements from clients and users on data projects Strong experience in Infrastructure as Code (IaC) and deploying infrastructure across environments Managing cloud infrastructure with a DevOps approach Handling and transforming various data types (JSON More ❯

JOB TITLE: Senior Data Engineer SALARY: £70,929 - £78,810 LOCATION: Edinburgh, Manchester, Leeds or Bristol HOURS: Full time WORKING PATTERN: Hybrid, 40% (or two days) in an office site About this opportunity At Lloyds Banking Group, we're moving to the next stages of our digital transformation, and it's our job within the Chief Security Office … CSO) to ensure that we keep our customers, colleagues and assets safe from threat. The CSO's Security Data Services lab does this by using data to build security insights by working collaboratively with our Data Management and Analytics teams. This is an exciting opportunity for a Senior Data Engineer to join … the Security Data Services lab to collaborate with the Data Operations, Software Engineers, Data Scientists, and Business SMEs to design, develop, integrate and test Data Product features on the product roadmaps to meet customer needs. What you'll do Engineer Data Products that are stable, scalable, performant, accessible, testable and More ❯

growing organisation? Do you have a passion for enabling people to perform at their best? If so, we want to hear from you. We are looking for a Data Engineer (Python) to join our Data Science and Technology team at BIT. As a Data Engineer you will be responsible for the technical delivery of … building and maintaining robust, scalable technology. You will play a crucial role in maintaining and enhancing Predictiv - our online research platform running at production scale - while delivering innovative data engineering solutions for client-facing projects. This is an exciting opportunity to use your technical skills while contributing to meaningful, high-impact projects. You will gain hands-on experience … platforms (e.g., AWS, GCP, or Azure). An understanding of DevOps concepts such as Infrastructure as Code, CI/CD, or MLOps. Knowledge of Python-based data pipeline frameworks (e.g. Airflow, Metaflow) and Python-based product development packages (e.g. FastAPI, Streamlit). An interest in developing or improving full stack product development skills. Who are we and More ❯

dynamic environment. About the Role: Full-Stack Development : Design, build, and maintain both front-end and back-end components for our SaaS security platform, with particular emphasis on data processing and analytics systems Data Engineering: Build and optimize data pipelines, ETL processes, and real-time data streaming systems to support security analytics … and intelligence. Data Architecture: Design and implement scalable data storage solutions, including databases, data warehouses, and distributed data systems. Security Focus : Work on security-related features, integrating cutting-edge security protocols, and building tools that help secure our customers' data and infrastructure. Collaboration : Work closely with other engineers, product managers … technologies, and best practices, and contribute ideas to the development of new security features. Key Qualifications: 1-2 years of experience in software development. Experience with data pipeline development and optimization (ETL/ELT processes). Solid understanding of distributed systems, databases,(PostgreSQL, Databricks, Clickhouse, ElasticSearch), and performance tuning. Familiarity with modern web frameworks and front-end More ❯

Prestigious opportunity for a Data Platform Engineer within a pioneering technology team based in Manchester. Following a period of significant growth, investment and a cloud transformation, we are inviting you to join our success story! As our Azure Data Platform Engineer, you will be responsible for:- Designing, building, and maintaining scalable data solutions on … Microsoft Azure. Designing and implementing scalable data pipelines using Azure Data Factory, Databricks, Synapse Analytics, and other Azure services. Leading technical workstreams and supporting project delivery. Acting as the subject-matter expert for cloud-based data engineering. Ensure data governance, security, and compliance standards are met. Monitoring and troubleshooting data platform performance and issues. Automate data ingestion, transformation, and integration processes. Document technical solutions and maintain best practices. If you possess a combination of some of the following skills, then LETS TALK! Proven experience with Azure data services (SQL, Data Factory, Data Lake, Synapse, Databricks, Azure SQL and Cosmos DB More ❯

Who We Are TetraScience is the Scientific Data and AI Cloud company. We are catalyzing the Scientific AI revolution by designing and industrializing AI-native scientific data sets, which we bring to life in a growing suite of next gen lab data management solutions, scientific use cases, and AI-enabled outcomes. TetraScience is the … in this vital new market, generating more revenue than all other companies in the aggregate. In the last year alone, the world's dominant players in compute, cloud, data, and AI infrastructure have converged on TetraScience as the de facto standard, entering into co-innovation and go-to-market partnerships: Latest News and Announcements TetraScience Newsroom In connection … EU. The ideal candidate should have expertise in customer and field enablement, adult learning principles, educational program creation, use case architecture and implementation, and a genuine passion for Data and AI technologies. The candidate should be driven by a desire to educate and inspire, and have the technical skills to quickly prototype new ideas and solutions. And finally More ❯

AWS Data Engineer - Manchester - Hybrid - £75,000 We're working with a global insurance company that's redefining the industry through technology. As a key end user of AWS, they're leveraging cloud-native solutions and advanced analytics to transform how they serve millions of customers. They're looking for Data Engineers who are passionate about … building scalable data platforms and want to be part of a forward-thinking team. You'll work on modern data pipelines, cloud infrastructure, and real-time analytics projects that directly impact business outcomes. As an AWS Data Engineer, you'll play a critical role in building and maintaining our cloud-based data infrastructure. You'll work closely with data scientists, analysts, and other engineers to design, develop, and deploy scalable data solutions on the AWS platform. The client operates with a hybrid working model, with two days per week in their Manchester office. Responsibilities: -Design, build, and maintain data pipelines using AWS services such More ❯

Lead Data Engineer - Manchester - Hybrid - £75k - £80k This is an excellent opportunity for a highly experienced data engineer to join a growing tech consultancy and contribute to the leadership of their data engineering practice. If you are an experienced and capable data engineer with a history of using Microsoft products and working … of clients to gather requirements and deliver solutions. Be willing to engage and assist in pre-sales activities, bids, proposals etc. Use key techniques such as Governance, Architecture, Data Modelling, ETL/ELT, Data Lakes, Data Warehousing, Master Data, and BI. Consistently utilise key processes in the engineering delivery cycle including Agile … and DevOps, Git, APIs, Containers, Microservices and Data Pipelines. Assist in building out, developing, and training the data engineering function. What do I need to apply Strong MS data engineering expertise (Fabric, ADF, ADL, Synapse, SQL) Expert use of Databricks Strong Python experience Consultancy experience Leadership experience My client are looking to book in More ❯

DATA ENGINEER/MANCHESTER/HYBRID/£60,000 - £67,000 PLUS BENEFITS WEX Europe Services Ltd are the owner of the Esso Card Fuel Card Portfolio, and with offices across Europe and the US are one of the Europe’s largest providers of fuel cards. As a Data Engineer at WEX, you’ll be responsible … for building and maintaining the bridge between our data and the rest of the organization. You’ll work with stakeholders to understand business requirements and then implement SQL-first transformation workflows to deploy analytics code. You’ll help ensure the integrity, reliability, and usability of data for stakeholders, so they can make critical data … are a critical thinker with strong analytical and problem-solving abilities. You are self-motivated and able to work independently with minimum supervision. We are seeking an experienced Data Engineer to play a critical role in the development of WEX's data & analytics capabilities. You will be part of an organization focusing on the development and More ❯

Location: Guernsey, Haywards Heath, Home Office (Remote) or Manchester Salary: from £70,000 depending on experience Department: Technology and Data We're 1st Central, a market-leading insurance company utilising smart data and technology at pace. Rapid growth has been based on giving our 1.4 million customers exactly what they want: great value insurance with an … design, through development and testing, to deployment, production, monitoring and continuous improvement optimising cutting-edge AI solutions across the business. One day you might be collaborating with our Data Scientists, Software Engineers and Architects on a new GenAI capability. The next, you're fine-tuning prompts or experimenting with Computer Vision and Natural Language Processing (NLP). You … control, and continuous integration. Develop and maintain automated frameworks for testing, validation, and monitoring of AI model accuracy, performance, and reliability. Establish best practices for AI data pipeline development and integration with infrastructure to support scalable deployment. Collaborate closely with Data Scientists, Data Engineers, Software Developers, and Architects for seamless execution of AI More ❯

Senior Data Scientist at AJ Bell (Sponsorship Available) AJ Bell is looking for an experienced and driven Senior Data Scientist to join its dynamic team in Manchester, United Kingdom. This is a fantastic opportunity to contribute to data-driven innovation within one of the UK's leading investment platforms. Visa sponsorship may be available … for exceptional candidates. The ideal candidate will have a minimum of 5 years' experience in data science or a related field, with extensive knowledge of advanced statistical analysis and machine learning algorithms. Proficiency in Python, R, and SQL is essential. As a Senior Data Scientist, you will play a key role in developing data … and enhance operational efficiency. Join AJ Bell and be part of a forward-thinking organisation that values expertise, innovation, and continuous learning in a collaborative environment. Position: Senior Data Scientist Job Type: Full-Time Location: Manchester About the Role Hackajob is partnering with AJ Bell to fill this position. Create a profile to be automatically considered for this More ❯

Data Engineer Hybrid Lancashire (2 Days Onsite per Week) Permanent | 37 Hours | £55,000£65,000 + Benefits Looking to use your data engineering skills … on meaningful projects? We're working with an established UK-based technology business that develops solutions to support critical national infrastructure. With demand growing and new projects in the pipeline, the company is expanding its data team and looking for a proactive, cloud-savvy Data Engineer. What Youll Be Doing: Designing and building scalable data pipelines across a modern cloud platform. Driving automation and ensuring data quality in complex data workflows. Contributing to the design and development of robust data architecture. Collaborating with internal teams to solve data challenges and support analytics. Sharing best practices and mentoring less experienced colleagues. What Youll Need: Hands-on More ❯

defence leaders to act with confidence in an increasingly complex world. Our trusted defence, security, and geopolitical information delivered through seamless digital platforms and system integrations-turns overwhelming data into clear, actionable intelligence and insight. By filling critical information gaps, Janes helps customers analyse threats, accelerate decisions, and stay ahead of emerging challenges. Job purpose: We are seeking … experienced Software Engineers with expertise in one of a number of identified technical areas, who have a strong background working with complex data, comprehensive experience in developing, high impact scalable systems, transforming and delivering complex datasets, and showing a high appreciation for data quality. You will be working as part of a multi-functional team based … DocumentDB/MongoDB RDF Graph database such as GraphDB ML/AI such as Sagemaker/Bedrock Search technologies such as SOLR or Opensearch/ElasticSearch Data pipeline engineering utilising cloud-based technologies (AWS) Write high quality clean, testable code, with a focus on incremental innovation. Take part in code and design reviews helping to improve overall More ❯

Data Engineer (AI-Driven SaaS plaform)(Python, Snowflake, Data Modelling, ETL/ELT, Apache Airflow, Kafka, AWS)Large-scale data environmentUp to £70,000 plus benefitsFULLY REMOTE UK Are you a Data Engineering enthusiast who thrives from designing and implementing robust ETL processes, highly scalable data structures and data pipelines within a truly enterprise-scale data processing environment? Do you want to work on an AI-Driven SaaS platform that's utilised by some of the biggest brands in the world? If so - look no further.This is your chance to help my client build their new Data Infrastructure and essentially help them to … leverage the power of data, working on large-scale data processing systems.They have an award-winning, enterprise-scale SaaS platform that was without a doubt, revolutionized its industry and is being utilised by some of the biggest companies and brand names in the WORLD. They have grown quickly and have truly made an impact in their More ❯

Data Engineer Location: Burnley, Lancashire Contract: Permanent - 37.5 hours per week Hybrid (2 days a week in office) Salary: £50-55,000 + performance bonuses + great benefits Are you a passionate Data Engineer looking to work on meaningful projects that make a difference to the environment? Fancy joining a friendly, inclusive team where your ideas … and growing organisation thats been a pioneer in water network monitoring for over 20 years. As the sector steps up to meet new environmental challenges, the demand for data-driven solutions is growing fast - and this team is at the forefront of it all. If youre excited by working with large, complex datasets, building scalable data pipelines, and helping to solve real-world problems, wed love to hear from you. What youll be doing: Designing and building pipelines in the data platform, with automation, quality and resilience at the core. Supporting the development of a scalable data architecture and engineering roadmap. Helping colleagues across the business solve data More ❯

Lead Data Engineer - Remote - UK About the Role: We're hiring on behalf of a fast-growing, tech-driven organisation looking for a Lead Data Engineer to take ownership of their data infrastructure. This is a fully remote role open to candidates based anywhere in the UK. You'll lead the design and development … of scalable data pipelines, mentor a small team of engineers, and work closely with cross-functional teams to deliver high-impact data solutions. Key Responsibilities: Architect and maintain robust data pipelines using AWS services (Glue, Lambda, S3, Airflow) Lead the migration and optimisation of data workflows into Snowflake. Collaborate with analysts, data scientists, and product teams to deliver clean, reliable data. Define and enforce best practices in data engineering and governance. Mentor junior engineers and contribute to a culture of technical excellence Requirements: Proven experience in a Lead or Senior Data Engineering role Strong hands-on expertise with AWS (Glue, Lambda, S3) Deep experience with Snowflake More ❯