1 to 25 of 68 Data Pipeline Jobs in the North West

Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data … proficiency, including experience with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience ...

member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. Job Title: Data Engineer Job Location: Preston – Hybrid- 1 day a week onsite. We offer a range of hybrid and flexible working arrangements – please speak to your recruiter about … modelling practices Designing and implementing ETL pipelines using SSIS for data integration to ensure ETL processes performs optimally Troubleshooting; identifying and fixing pipeline bottlenecks and data issues Understanding of data governance, data quality, and best practices The Data Management ...

data engineer volunteers can support with: MapAction external projects that require a Data Engineer skillset (e.g.: data pipeline automation, hazard exposure platforms, multi-risk models, conflict incident reporting, etc...) Work with external partners such as UN agencies, International NGOs and Academic Institutions Conduct … automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Assist in the automation, provision and management of development environments, source-code, build-tools, pipelines and deployments Support ...

decide faster, and act with confidence across complex domains. We deliver situational awareness and information advantage across C5ISR, fusing cyber, geospatial, and kinetic data into an operational picture. This transforms fragmented data into a shared understanding of the battlespace, delivering actionable insight when it matters most. … distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud‐native data platforms, with strong expertise in AWS and native data services. Familiarity with Infrastructure ...

Broster Buchanan are partnered global business within the food industry with a turnover of £8Bn+ who are looking for a Fabric Data Engineer to join as they set up an exciting new project team within the group. This would suit an experienced Azure Data Engineer with … offer. Responsibilities: Help set up and deliver the future direction of strategy and approach to the management, storage and usage of data Collaborate with their wider technology and business teams to ensure goals and visions are aligned and achieved Help with setting up the data warehousing ...

Data Architect £Up to £120,000 GBP Competitive Bonus + Car Allowance Hybrid WORKING Location: London; Bristol; Manchester, Central London, Greater London - United Kingdom Type: Permanent Data Architect London | Manchester | Bristol | Hybrid | Permanent Our client is a leading global technology and consulting organisation , recognised … environments. Due to the nature of these programmes, active SC clearance is required for this role . This opportunity sits within a growing Data & AI Architecture capability , focused on delivering complex, high-impact programmes at the heart of government. The role combines deep technical architecture expertise with leadership ...

Overview This is an exciting opportunity for a technically minded Data Engineer to join a growing and innovative organisation working at the forefront of data-driven solutions. Youll play a key role in building, maintaining, and optimising data systems that support operational performance … reporting. This role offers exposure to modern data tools, cloud technologies, and large-scale datasets, making it ideal for someone looking to develop a career in data engineering. Data Engineer Key Responsibilities Data Integration & ETL Design, develop, and optimise ETL processes ...

part of this position. Main Responsibilities Conduct and automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Ensure find-ability. availability, scalability, reproducibility, accessibility, interoperability and reusability … Jupyter Notebooks Experience working with geospatial data (raster and vector) Databases (NoSQL, SQL, Graph Dbs, Principles of RDBs) Data pipeline design skills Knowledge of cloud platforms like GCP, AWS or Azure Experience and understanding of CI/CD pipelines Collaborative code management using Git & GitHub ...

automation in data engineering — this is a small team that needs to move fast, and we expect the use of AI‐assisted pipeline development and management to be embedded in how we work Data Governance Lead the Data Governance function from a standing … alongside a technology partner or platform team (similar to our Ingenuity relationship) Demonstrated use of AI in data engineering contexts — AI‐assisted pipeline development, automated quality monitoring, or similar Strong leadership and stakeholder management skills — able to influence at Director and C‐suite level Experience ...

driving the evolution of our enterprise data engineering capabilities across our cloud and legacy environments. Your remit spans cloud platform engineering, pipeline automation, real‐time data processing, curation, and the enablement of business intelligence, data science and AI teams. You will … concepts clearly and frame engineering decisions in terms of business impact. Ensure data platforms meet regulatory, security, and compliance requirements, including secure pipeline design, access control, and lineage. Embed good governance practices across metadata, cataloguing, quality, and monitoring. Define and maintain data engineering standards, documentation ...

data stack, connect APIs and pipelines into coherent systems, and take ownership of the full journey from initial brief through to a pipeline running reliably in production. If you are the kind of engineer who takes pride in clean pipelines, fast queries, and infrastructure that just works … it. What you will be doing Owning BI infrastructure end-to-end, from initial brief and requirement gathering through to data pipeline development across APIs, ETL, ELT and SQL Working alongside Data Analyst colleagues to help them generate impactful business insights and analysis Taking ...

Title: Data Engineer Location: Greater Manchester (Hybrid - 3 days per week onsite) Salary: £60,000 - £70,000 + Bonus + Excellent Benefits We're working with a well-established, market-leading organisation that is investing heavily in its data and technology capabilities. They are looking … appoint an experienced Data Engineer to lead the development of a modern cloud-based data platform that will support reporting, analytics and business decision-making across the organisation. This is an exciting opportunity to join a growing business undergoing digital transformation, where you'll have ...

belonging for all our colleagues is central to our journey as a global business. Who You Are/Your Next Challenge The Data Engineer plays a critical role in enabling Finance & Treasury to deliver accurate, timely, and insight‐driven reporting by building scalable data solutions, maintaining … core data infrastructure, and supporting the modernisation of Finance systems. This role is responsible for engineering robust data pipelines, managing datasets across the Microsoft Fabric platform, and ensuring Finance has reliable, consistent, and governed data to support transformation, automation, and improved operational efficiency. ...

business expands, we are looking for ambitious, talented people to join us. We are looking for a technically proficient Senior Data Engineer to join our growing Data team. Your primary focus will be on ensuring data quality, stability, and reliability — from the moment data arrives in its rawest form to when it is used in decision-making dashboards and customer-facing reports. You will optimise the transformation pipeline from start to finish, guaranteeing that datasets are robust, tested, secure, and business-ready. Our data platform is built using Databricks, with ...

Data Platform & Analytics Lead (Microsoft Fabric/AI Enablement) Manchester or Leeds (hybrid working available) The role: The Data & Analytics Lead owns my clients' enterprise data platform and is responsible for ensuring it supports analytics, AI, and machine learning use cases in a structured … governed, and scalable way. This includes end-to-end accountability for data ingestion, transformation, lakehouse design, semantic modelling, and the development of reliable data products. The role ensures business data is accessible, consistent, and trusted, with clearly defined metrics that enable high-quality analysis ...

Data Engineer Join a growing data engineering function and help build scalable, cloud-based data products that support data-driven decision making across the organisation. The Opportunity This is an exciting opportunity for a Data Engineer to work with modern … cloud technologies while contributing to the design, development, and delivery of enterprise data products. Working alongside Product Managers, Architects, Principal Engineers, and fellow Data Engineers, you will help create robust, high-performing data solutions that transform raw data into valuable business insights. ...

Title: AWS Data Engineer Contract: 6 Months Location: Remote, UK Skills: Advanced knowledge of data management tools including SQL/DBMS, MongoDB, Hadoop and/or other big data technologies. Advanced programming skills in Java, Python, R, C++, C#, etc. Knowledge of statistical … dynamics and corporate functions to help clients create the most value from data. Must Have: Enterprise Knowledge Data Management, Data pipeline, Python, SQL, AWS Centralising data stored in RDBMS, Data Ingestion, and Uploading data into the database. Education: Bachelor ...

Data Engineer - Defence ££60,000 - £90,000 GBP Hybrid WORKING Location: Manchester;Bristol;London;Newcaslte , Manchester, North West - United Kingdom Type: Permanent Data Engineer Newcastle | Manchester | Bristol | London | Hybrid | Permanent Our client is a specialist technology consultancy delivering high-impact engineering across defence and complex environments … engineering within defence and secure environments Modern cloud-based data platforms and distributed systems End-to-end data pipeline design and optimisation High-assurance, real-world engineering challenges Embedded delivery within client environments Why Join: Work on mission-critical data systems ...

myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. About the job Job title: Data Engineer Location: Birkenhead - Hybrid working Working hours: 35 hours per week About HSB HSB (UK and Ireland) is a leading specialist provider of engineering, technology … components Understanding of data modelling, schema design and performance optimisation Familiarity with ETL/ELT concepts and data pipeline development Experience using Git and version control practices Exposure to Azure data tools such as Azure SQL, Data Factory ...

complexity manageable and insight actionable supporting everything from premium and ads growth to forecasting, experimentation, and global reporting. As engineers in the Financial Data Platform team, we turn messy, fragmented realities into clean, reusable foundations. We build core datasets that represent key financial domains like Premium … Royalties. We create libraries and tools that empower others to produce and trust financial data at scale. We collaborate deeply with Finance, Product, and Data teams to unlock clarity and drive Spotify s ambitions forward. We are looking for engineers who are excited to shape ...

Apprentice Data Engineer Location: Bristol, Nottingham, Birmingham, Leeds, Oldham, Newcastle, Uxbridge Salary: £30,485 per annum Vacancy Type: Fixed term apprenticeship, 2 years Closing Date: 12th of July 2026 This Apprenticeship combines working and learning, in a great opportunity where you’ll earn a salary, get workexperience … finish the apprenticeship with a Level 5 Qualification in Data Engineering – which you won’t have to pay a penny for! We will provide a fully paid 24 month Apprenticeship to help you gain a Level 5 qualification specialising in Data Engineering and provide hands ...

Data Engineer Manchester £45,000 About the Company A fast-growing, family-oriented organisation operating across multiple locations in the UK with a strong focus on people, development and long-term career growth are looking for a Data Engineer to join their growing team. As their … business continues to expand, they are investing heavily in data and technology to support smarter decision-making and innovation across the organisation. This is an excellent opportunity for someone with some hands-on experience who is eager to further develop their career in data engineering, with ...

Data Warehouse Architect £75,000+ benefits Bury (Hybrid) We are seeking a Data Warehouse Architect to design, establish, and govern a modern enterprise data platform. This role will bridge complex raw data and business insights by creating scalable, high-performance lakehouse … data warehouse architectures. You will play a key role in consolidating data assets, eliminating silos, and ensuring a single, source of truth to support advanced analytics and AI. Data Warehouse Architect Responsibilities: Design and implement enterprise-wide data architecture across Microsoft Fabric ...

Data Engineer Remote Working/London circ £60,000 - £70,000 Experienced Data Engineer required for this remote working/hybrid position based in West London (near Feltham, Hanworth, Hounslow etc). Ideally the successful candidate will be able to work on site once or twice … client would also consider candidates looking for a remote working role as long as they can be onsite once a month. The Data Engineer will join a small team and play a key role in developing and managing data integration projects as our client transitions data ...

Were looking for a Head of Integration, Data & GenAI Engineering to lead a critical engineering capability within Technology Services at AJ Bell. This role brings together leadership across three important parts of our technology landscape: integration engineering, data engineering and Generative AI engineering. Reporting … Director of IT Strategy & Architecture, you will lead through three senior technical leaders: Senior Engineering Manager (EA), Senior Data Engineering Manager, and Software Engineering Team Lead (GenAI). You will help shape the platforms, capabilities and engineering practices that support AJ Bells long-term strategy. This ...