1 to 25 of 195 Data Pipeline Jobs in the North of England

Engineer, Lead Data Engineer, Data Engineer, ETL, AWS, AWS Glue, AWS Lambda, AWS S3, AWS Redshift, Amazon EMR, Python, SQL, Pipeline Orchestration, PySpark, data warehousing, data lakehouse, data architecture, Agile, Senior Data Engineer, Lead Data … proficiency, including experience with AWS services such as AWS Glue, AWS Lambda, AWS S3, AWS Redshift, and Amazon EMR. Proven expertise in Python, SQL, Pipeline Orchestration and PySpark is also expected, as is experience with data lakehouse architectures and data warehousing solutions. Agile experience ...

Title: Data Engineer Location: UK Job Type: Full-time About Scrumconnect: Scrumconnect is a leading force in technology consultancy, proudly contributing to over 20% of the UK’s most significant citizen-facing public services. Our award-winning team has made a substantial impact, delivering more than 64 services … Science, Architecture, etc.) to identify emerging and cross-community issues affecting your role and escalate as necessary. Describe technical, data, pipeline, and production issues to colleagues of different specialisms. Communicate within the team and across teams to monitor expectations around delivery of data ...

Title: Data Engineer Location: UK Job Type: Full-time About Scrumconnect: Scrumconnect is a leading force in technology consultancy, proudly contributing to over 20% of the UK’s most significant citizen-facing public services. Our award-winning team has made a substantial impact, delivering more than 64 services … Science, Architecture, etc.) to identify emerging and cross-community issues affecting your role and escalate as necessary. Describe technical, data, pipeline, and production issues to colleagues of different specialisms. Communicate within the team and across teams to monitor expectations around delivery of data ...

Senior Data Engineer Remote UK | Occasional Travel Senior Data Engineer | Azure | Microsoft Fabric | Data Platform | Python | SQL | Power BI | AI & Analytics 167 Solutions are partnering with a global organisation embarking on a significant data transformation programme. As investment in analytics … business intelligence continues to grow, we are seeking a Senior Data Engineer to play a key role in shaping and delivering a modern enterprise data platform. This opportunity offers the chance to work on large-scale data initiatives, modern cloud technologies and next-generation ...

from the landing zone into the Bronze layer, working in partnership with the Development and Application Management team. Design and optimise ETL/ELT pipeline frameworks using Databricks, Delta Lake and Azure Data Factory. Ensure Silver and Gold layer data products are fit for purpose … modelling — conceptual, logical and physical. Data warehousing and data lake architecture for high‐performance analytics. ETL/ELT pipeline development and optimisation to support large‐scale data processing. Data integration across structured and unstructured sources, ensuring high availability. Metadata ...

solutions for performance, reliability, and scalability, ensuring efficient data processing and storage. Implement and manage orchestration workflows to automate data pipeline scheduling monitoring and error handling. The Requirements As a Data Engineer you will play a crucial role in transforming data … designing and delivering data pipelines using industry-standard tools and cloud technologies. Knowledge of data engineering concepts, data pipeline development, ETL processes, data lake and data warehousing technologies. Proficiency in working with at least one major cloud data ...

decide faster, and act with confidence across complex domains. We deliver situational awareness and information advantage across C5ISR, fusing cyber, geospatial, and kinetic data into an operational picture. This transforms fragmented data into a shared understanding of the battlespace, delivering actionable insight when it matters most. … distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud‐native data platforms, with strong expertise in AWS and native data services. Familiarity with Infrastructure ...

decide faster, and act with confidence across complex domains. We deliver situational awareness and information advantage across C5ISR, fusing cyber, geospatial, and kinetic data into an operational picture. This transforms fragmented data into a shared understanding of the battlespace, delivering actionable insight when it matters most. … distributed processing technologies (e.g. Apache Spark, Hadoop). Experience designing and maintaining data lakes, distributed storage platforms, and data pipeline orchestration tools. Experience working with cloud‐native data platforms, with strong expertise in AWS and native data services. Familiarity with Infrastructure ...

Broster Buchanan are partnered global business within the food industry with a turnover of £8Bn+ who are looking for a Fabric Data Engineer to join as they set up an exciting new project team within the group. This would suit an experienced Azure Data Engineer with … offer. Responsibilities: Help set up and deliver the future direction of strategy and approach to the management, storage and usage of data Collaborate with their wider technology and business teams to ensure goals and visions are aligned and achieved Help with setting up the data warehousing ...

Broster Buchanan are partnered global business within the food industry with a turnover of £8Bn+ who are looking for a Fabric Data Engineer to join as they set up an exciting new project team within the group. This would suit an experienced Azure Data Engineer with … offer. Responsibilities: Help set up and deliver the future direction of strategy and approach to the management, storage and usage of data Collaborate with their wider technology and business teams to ensure goals and visions are aligned and achieved Help with setting up the data warehousing ...

Broster Buchanan are partnered global business within the food industry with a turnover of £8Bn+ who are looking for a Fabric Data Engineer to join as they set up an exciting new project team within the group. This would suit an experienced Azure Data Engineer with … offer. Responsibilities: Help set up and deliver the future direction of strategy and approach to the management, storage and usage of data Collaborate with their wider technology and business teams to ensure goals and visions are aligned and achieved Help with setting up the data warehousing ...

Data Architect £Up to £120,000 GBP Competitive Bonus + Car Allowance Hybrid WORKING Location: London; Bristol; Manchester, Central London, Greater London - United Kingdom Type: Permanent Data Architect London | Manchester | Bristol | Hybrid | Permanent Our client is a leading global technology and consulting organisation , recognised … environments. Due to the nature of these programmes, active SC clearance is required for this role . This opportunity sits within a growing Data & AI Architecture capability , focused on delivering complex, high-impact programmes at the heart of government. The role combines deep technical architecture expertise with leadership ...

part of this position. Main Responsibilities Conduct and automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Ensure find-ability. availability, scalability, reproducibility, accessibility, interoperability and reusability … Jupyter Notebooks Experience working with geospatial data (raster and vector) Databases (NoSQL, SQL, Graph Dbs, Principles of RDBs) Data pipeline design skills Knowledge of cloud platforms like GCP, AWS or Azure Experience and understanding of CI/CD pipelines Collaborative code management using Git & GitHub ...

part of this position. Main Responsibilities Conduct and automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Ensure find-ability. availability, scalability, reproducibility, accessibility, interoperability and reusability … Jupyter Notebooks Experience working with geospatial data (raster and vector) Databases (NoSQL, SQL, Graph Dbs, Principles of RDBs) Data pipeline design skills Knowledge of cloud platforms like GCP, AWS or Azure Experience and understanding of CI/CD pipelines Collaborative code management using Git & GitHub ...

part of this position. Main Responsibilities Conduct and automate data collection and processing scripts Create and maintain optimal, performant data pipeline architectures, both internally and externally Propose improvements to infrastructure, schemas, platforms, toolchains and configurations Ensure find-ability. availability, scalability, reproducibility, accessibility, interoperability and reusability … Jupyter Notebooks Experience working with geospatial data (raster and vector) Databases (NoSQL, SQL, Graph Dbs, Principles of RDBs) Data pipeline design skills Knowledge of cloud platforms like GCP, AWS or Azure Experience and understanding of CI/CD pipelines Collaborative code management using Git & GitHub ...

Description As part of our continued growth and investment in our data and analytics capabilities, we’re recruiting for an experienced Senior BI Developer to join our dynamic direct-to-consumer (D2C) business. This is a key opportunity to play a central role in shaping how data … decisions, and support ongoing business success. As the Senior BI Developer, you will play a pivotal role in designing, building, and maintaining the data assets that underpin our reporting, analytics, and customer marketing capabilities. You’ll work closely with stakeholders to define requirements, develop best-in-class data ...

Azure Data Engineer Manchester(Hybrid) You'll design and build scalable, automated data pipelines that bring together information from across the organisation. It's a hands-on engineering role covering everything from data ingestion to transformation, modelling and delivery, all within a modern Azure … environment. You'll report to the Lead Data Engineer and join our Data Engineering Team, working alongside architects, analysts, testers and agile delivery squads. Responsibilities: * Build and maintain scalable, automated data pipelines using Azure tools like Data Factory, Databricks and Logic Apps ...

learning allowances, and, of course, our annual Airalo Away retreat. Learn more about our benefits here; Hi, I'm Andra, Director of Data at Airalo! Our team works across the full data ecosystem, from collection to insights activation, ensuring that every piece of data … distributed data processing frameworks (Apache Spark, Flink). Hands-on experience using AI tools to accelerate engineering workflows - code generation, code review, pipeline debugging, or documentation Strong coding experience in Python (and/or Scala) and advanced SQL across relational and non-relational databases. Experience implementing ...

learning allowances, and, of course, our annual Airalo Away retreat. Learn more about our benefits here; Hi, I'm Andra, Director of Data at Airalo! Our team works across the full data ecosystem, from collection to insights activation, ensuring that every piece of data … distributed data processing frameworks (Apache Spark, Flink). Hands-on experience using AI tools to accelerate engineering workflows - code generation, code review, pipeline debugging, or documentation Strong coding experience in Python (and/or Scala) and advanced SQL across relational and non-relational databases. Experience implementing ...

learning allowances, and, of course, our annual Airalo Away retreat. Learn more about our benefits here; Hi, I'm Andra, Director of Data at Airalo! Our team works across the full data ecosystem, from collection to insights activation, ensuring that every piece of data … distributed data processing frameworks (Apache Spark, Flink). Hands-on experience using AI tools to accelerate engineering workflows - code generation, code review, pipeline debugging, or documentation Strong coding experience in Python (and/or Scala) and advanced SQL across relational and non-relational databases. Experience implementing ...

better outcomes for our patients and communities, work to get the best possible value for taxpayers, and drive improvement across the NHS. The Data Services division of Data and Analytics sub directorate's vision is to be the leading data services provider for health … social care across England through creating innovative and high-quality data products and services that give our customers the knowledge and capabilities required to deliver the best outcomes for citizens. The Data Engineering team work with a number of key delivery programmes including but not limited ...

better outcomes for our patients and communities, work to get the best possible value for taxpayers, and drive improvement across the NHS. The Data Services division of Data and Analytics sub directorate's vision is to be the leading data services provider for health … desirable Skills and Experience Essential Can deliver well-scoped work independently Applies standard design patterns and engineering practices Debugs and resolves data pipeline issues Communicates effectively with technical and semi-technical stakeholders. Values Essential Proactive, team-oriented, willing to learn Open to feedback and continuous improvement. Disclosure ...

Role: Data & ML Architect Location: Newcastle Upon Tyne Levels: Associate Manager Please Note: Due to the nature of client work you will need to go through a Security Clearance (Including BPSS) process as part of this role, which requires 5+ years UK address history at the point … require you to work from our Newcastle, Cobalt Business Park office for a minimum of 3 days per week. Role Overview As a Data & ML Architect, you will lead the design and architecture of scalable, enterprise-grade data platforms and advanced machine learning solutions. You will ...

currently seeking a Data Warehouse Architect who will be responsible for establishing, optimising, and governing a unified enterprise data platform that enables data‐driven decision‐making. Acting as a bridge between complex raw data and meaningful business insight, the role involves designing … Microsoft Fabric (OneLake) and Databricks (Unity Catalog) architecture. Experience building scalable lakehouse and warehouse solutions (Databricks SQL, Fabric SQL). Strong data pipeline development using Databricks Workflows, DLT, and Fabric Data Factory. Knowledge of Delta Lake and cross‐platform data strategies to reduce ...

business expands, we are looking for ambitious, talented people to join us. We are looking for a technically proficient Senior Data Engineer to join our growing Data team. Your primary focus will be on ensuring data quality, stability, and reliability — from the moment data arrives in its rawest form to when it is used in decision-making dashboards and customer-facing reports. You will optimise the transformation pipeline from start to finish, guaranteeing that datasets are robust, tested, secure, and business-ready. Our data platform is built using Databricks, with ...