1 to 25 of 30 Data Pipeline Jobs in West Yorkshire

allows customers to make a single monthly payment, to receive brand new equipment, and have maintenance costs taken care of. Role overview The Data Engineer will work within the data team to assist in the design, implementation, and maintenance of our Azure-based data warehouses. The data engineer will be involved in all aspects of the data warehouse, covering integration to new data sources, design and creation of new data pipelines as well as the optimization and maintenance of existing ones. They will also … Azure based data warehouse technologies such as Azure Data Factory, Analysis Services, SQL server and Azure Synapse. Data Pipeline Creation: Build and optimize ETL/ELT data pipelines using Azure Data Factory, Databricks, or similar services to ensure data More ❯

with length of service Sick Pay Increasing with length of service The Role: DataOps Engineer Job Description We are seeking a senior Data DataOps Engineer to serve as our first DataOps specialist in a growing team of Data Engineers and DevOps professionals. In this pivotal role … you will focus on operationalising and automating our data lifecycle to ensure that data workflows perform with reliability and efficiency. You will integrate CI/CD data pipelines, streamline deployment processes, enforce robust data governance, and optimise operational costs within our Microsoft …/CD data pipelines and strong programming skills in Python, SQL, Bash, and PySpark for automation. Strong aptitude for data pipeline monitoring and an understanding of data security practices such as RBAC and encryption. Implemented data and pipeline observability dashboards More ❯

Job Title: Data Engineer Location: Leeds ( 1 day every 2 weeks) Department: Data & Analytics Salary: £70,000 - £80,000 About Us One of the UK’s leading services for selling items in one simple, hassle-free process. With a mission to promote sustainability and support a … circular economy, as they expand into new markets, they are looking for a highly skilled and motivated Data Engineer to join the dynamic team and help elevate their data infrastructure and analytics capabilities. If you have experience with Snowflake, digital marketing analytics, and are familiar with … data warehousing as well as what best practice looks like when undertaking data engineering. Key Responsibilities Data Architecture & Pipeline Development: Design, implement, and maintain robust and scalable data pipelines, ensuring seamless integration and flow of data across multiple systems. More ❯

Senior Data Engineer - Data Infrastructure and Architecture: C-4 Analytics C-4 Analytics is a fast-growing, private, full-service digital marketing company that excels at helping automotive dealerships increase sales, increase market share, and lower cost per acquisition. We are currently hiring for a Senior … Data Engineer as we look to expand our team and support our growing roster of local and national clients. If you are unable to complete this application due to a disability, contact this employer to ask for accommodation or an alternative application process. Who We're Looking For … growing AI team, you'll play a critical role in the orchestration of Intelligent Systems. Working with AI related orchestration and data pipeline technologies to turn information into insights using multiple platforms. We're not just processing data-we're transforming it into organizational intelligence. More ❯

Data Engineering Vacancies - Mid to Lead Level - Leeds I am working with a brand new client, and end user based in Leeds who are building an exciting data practice, doing some highly advanced work in the data space. Salary and Benefits Highly comptetitive salary … days per week in Leeds office Competitive annual leave package And many more Role and Responsibilities Design and manage Snowflake-based data pipelines, ensuring data flows seamlessly from multiple sources into the data platform. Collaborate with business stakeholders and data analysts to … identify data requirements and deliver the appropriate solutions. Ensure the integration of structured and unstructured data sources into Snowflake, optimizing data models for reporting. Work with data analysts to support their data needs and optimize semantic models for Snowflake-based More ❯

Senior Data Engineer Hybrid 2x per week (Leeds, West Yorkshire) Salary to £60,000 + benefits About the Role: Our client, a tech for good organisation, is investing heavily in digital transformation and building a cloud-first, data-led approach to support major environmental goals. Think … scalable pipelines, secure platforms, and data systems that directly contribute to a greener future. As a Senior Data Engineer, you'll play a key role in designing and delivering robust, cloud-based data infrastructure. You'll work with large datasets, build scalable pipelines, and … help ensure data is high quality, secure, and accessible, while mentoring more junior engineers and working closely with stakeholders across the business. Key Responsibilities: Develop scalable and secure data architectures ensuring optimal data flow Cloud data platform development, data modelling More ❯

My client is based in the Leeds area are currently looking to recruit for an experienced Data Engineer to join their Data & AI Consulting team. They are a specialist Tech organisation, that are at the forefront of engineering practices. They are currently going through a period … of growth and are looking for an experienced Data Engineer to join their team. They only recruit the "best" talent and have a diverse workforce. Key Responsibilities: Design, build, and manage data pipelines in Snowflake, ensuring seamless data flow from diverse sources into the … data platform. Collaborate with business stakeholders and data analysts to gather data requirements and deliver effective solutions. Integrate structured and unstructured data sources into Snowflake, optimising data models for analytics and reporting. Support data analysts by optimising semantic More ❯

Snowflake Data Engineer I am working with a hugely successful organisation based in Leeds who are looking for a Snowflake Data Engineer to join their newly formed data practice. You will join a business to work on their exciting Greenfield data platform … project, where you will have the opportunity to create the data warehouse from scratch and shape the ongoing direction of the data platform. You will work to build and maintain a robust and scalable data platform and will support the organisation's BI and … analytics goals. You will work closely with developers to transform data ready for reporting purposes to deliver accurate information to key business stakeholders. As part of this role you will be responsible for - Design and develop data pipelines ensuring the successful integration of data More ❯

I'm looking for a Data Quality & Governance Manager to join a global business in Leeds - they are building a modern data platform, using cutting-edge technologies including Snowflake and Microsoft Azure. You'll join their newly established Data Team, based in their modern … fast-paced, forward-thinking and entrepreneurial environment. In this brand-new role, you will lead on the development and implementation of frameworks for data quality, governance, and master data management (MDM), ultimately shaping how data is trusted and used across the organisation. You'll … work closely with Data Engineers, to ensure the effective management of data warehouse architecture and data pipelines, using a combination of Snowflake and Microsoft Azure technologies. You'll also look at implementing data governance tools and platforms (e.g. Purview), and ensure the More ❯

I'm looking for a Data Quality & Governance Manager to join a global business who are building a modern data platform, using cutting-edge technologies including Snowflake and Microsoft Azure. You'll join their newly established Data Team, based in their modern office space … fast-paced, forward-thinking and entrepreneurial environment. In this brand-new role, you will lead on the development and implementation of frameworks for data quality, governance, and master data management (MDM), ultimately shaping how data is trusted and used across the organisation. You'll … work closely with Data Engineers, to ensure the effective management of data warehouse architecture and data pipelines, using a combination of Snowflake and Microsoft Azure technologies. You'll also look at implementing data governance tools and platforms (e.g. Purview), and ensure the More ❯

our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the More ❯

Role: Data Engineering Manager Salary: £65,000- £70,000 per annum Location: Leeds (One day on site). VIQU have partnered with a leading supply chain organisation who are looking to expand their data teams. The Data Engineering Manager will manage a team of … is preferred but open to Azure.GCP experience also), utilising Python and SQL, contribute to CI/CD pipelines, strategy development, cost optimisation and data governance frameworks. Job duties of the Data Engineering Manager: Manage a team of eight engineers, helping to mentor and coach the team. … coaching similar sized teams. Deep knowledge of the Databricks platform. Hands on Python development experience. SQL optimisation. Experience with large scale data pipeline optimisation. Experience with Streaming and Batch Spark workloads. Strong people management skills. Role: Data Engineering Manager Salary: £65,000- £70,000 per More ❯

Title: Azure Data Engineer Location: Fully remote (working UAE hours) Salary: 5k GBP per month Are you a data expert with a passion for building scalable, cloud-first data pipelines? We’re hiring an experienced Azure Data Engineer to join a fast … growing team focused on developing a centralised, enterprise-grade data warehouse in the Azure cloud. This is a full-time remote role open to candidates located within GMT +1 to +6 time zones . If you thrive in fast-paced environments, take extreme ownership of your work, and … Creating and maintaining secure, scalable data pipelines and ingestion workflows Leveraging Azure Synapse Analytics and ADF to deliver robust ELT solutions Ensuring pipeline execution, availability, and reliability across platforms Owning physical data models and collaborating with BI teams on reporting layers Supporting data More ❯

I'm looking for a Data Engineer with Snowflake experience to join a global business who are embarking on a greenfield project to build a brand-new Snowflake Data Warehouse. They are building a modern, AI-ready data platform, which allows for better decision … both themselves and their customers, with a tech stack including Snowflake, dbt, Microsoft Azure and Power BI. You'll join their newly established Data Team, based in their modern office space in Leeds - you'll spend at least 3 days per week in the office to collaborate with … in what's a very fast-paced, forward-thinking and entrepreneurial environment. The purpose of this role is to build and maintain the data architecture that supports various business intelligence initiatives. This will include designing and managing Snowflake data pipelines, ensuring data flows seamlessly More ❯

Data Quality and Governance Manager - Hybrid - Leeds - £65,000 My client is looking for a Data Quality & Governance Manager to shape how data is trusted and used throughout their organisation. They are looking for someone to lead efforts in ensuring the consistency, reliability, and … governance of data. This role will work closely with business and technical teams to develop and implement frameworks for data quality, governance, and master data management, using Snowflake and Microsoft Azure. Requirements: Experience in data governance and data quality management Strong knowledge … of data warehouse architecture and cloud platforms, especially Snowflake and Microsoft Azure. Proven experience designing and managing data pipelines in cloud environments with integrated data governance principles. Hands-on experience with data migrations Strong experience in working with data engineers More ❯

Data Quality and Governance Manager - Hybrid - Leeds - £65,000 My client is looking for a Data Quality & Governance Manager to shape how data is trusted and used throughout their organisation. They are looking for someone to lead efforts in ensuring the consistency, reliability, and … governance of data. This role will work closely with business and technical teams to develop and implement frameworks for data quality, governance, and master data management, using Snowflake and Microsoft Azure. Requirements: Experience in data governance and data quality management Strong knowledge … of data warehouse architecture and cloud platforms, especially Snowflake and Microsoft Azure. Proven experience designing and managing data pipelines in cloud environments with integrated data governance principles. Hands-on experience with data migrations Strong experience in working with data engineers More ❯

Data Engineer An exciting opportunity has arisen for a skilled Data Engineer to support a government department in transitioning a successful proof-of-concept (PoC) business intelligence and data warehousing solution into a fully operational, production-ready platform. Over a four-year period, a … options aligned with the departments long-term digital strategy. Through agile delivery and close collaboration, the organisation significantly improved its understanding of internal data assets and the Microsoft technology ecosystem, enhancing its reporting capabilities and data-driven decision-making. The next phase of this programme will … Factory (ADF): Incremental creation of data dimensions to optimise processing, reduce overheads, and improve cost efficiency across compute, storage, and networking. Enhancing pipeline security by replacing basic authentication with managed identities or service principals. Introducing parallelism into pipeline workflows to improve performance and scalability. Refactoring existing More ❯

our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse the... More ❯

Senior Data Engineer – Assistant Manager Base Location: Leeds based (Hybrid – 3 days per week in office) www.kpmgcareers.co.uk/experienced-professional/#LeBlender.OfficeLocations As a result of the work that we do, we require applicants to hold or be capable of obtaining UK National Security Vetting, the requirements for … planet or everything in-between. In a world where tech is always changing, so are we. Why Join KPMG Curve as a Senior Data Engineer Progression at pace. Innovation, led by constant learning. Work that excites you with every twist and turn. It all starts at KPMG Curve. … one Data Platform Technology such as Quantexa, Palantir and DataBricks Knowledge of test automation frameworks and ability to automate testing within the pipeline To discuss this or wider Technology roles with our recruitment team, all you need to do is apply, create a profile, upload your CV More ❯

Analytics/Data Engineer ?? Location : Leeds - 2 days per week in the office ?? Salary: Up to £80,000 per annum + Benefits ?? Help Build a Brand-New Data Function from the Ground Up We're working with a fast-growing, tech-enabled business that's leading … off a huge transformation in how they use data. This is your chance to join at the very beginning of a newly formed data team, helping to revamp and modernise how data is collected, processed, and used to drive smarter decisions across marketing, operations, and beyond. … As the first Data Engineer on the team , you'll be instrumental in designing the foundation of this transformation - working directly with senior stakeholders and technical leads to create scalable, future-proof data infrastructure. ?? What You'll Be Doing Building and maintaining data pipelines More ❯

Location: Leeds City Centre + Hybrid Working Job Type: Full-time Team: Product Reports to: Head of Product Development About Us: We use data science and AI to help retailers make profitable pricing, marketing and retail media decisions. Put simply, we help retailers sell the right products, to … a business intelligence development or related role Excellent SQL skills Good understanding of relational and analytical database architectures and methodologies Experience of cloud data warehousing solutions e.g. Snowflake, BigQuery, Redshift Good hands-on experience with at least one Business Intelligence platform Strong analytical and problem-solving skills Excellent … the ability to explain technical details to non-technical stakeholders Comfortable with occasional client interaction in a professional and supportive capacity Great UX & data visualisation skills The Role: Design, develop and maintain BI solutions including dashboards, reports and data models Produce visually engaging deliverables for consumption More ❯

Job Title: Lead Data Engineer Base Location: Leeds based (Hybrid – 3 days per week in office) www.kpmgcareers.co.uk/experienced-professional/#LeBlender.OfficeLocations As a result of the work that we do, we require applicants to hold or be capable of obtaining UK National Security Vetting, the requirements for … planet or everything in-between. In a world where tech is always changing, so are we. Why Join KPMG Curve as a Lead Data Engineer Progression at pace. Innovation, led by constant learning. Work that excites you with every twist and turn. It all starts at KPMG Curve. … science applications Ability to use wide variety of open-source technologies Knowledge of test automation frameworks and ability to automate testing within the pipeline To discuss this or wider Technology roles with our recruitment team, all you need to do is apply, create a profile, upload your CV More ❯

gather requirements, driving projects from inception to deployment. You will leverage your expertise across object-oriented software engineering, quantitative modelling, cloud computing, and data analysis to help improve the models underpinning our most business-critical cashflow and pricing engines. You will come up with ad-hoc experimental analysis … tests and scope new projects Create and update technical documentation Explore and clean datasets required for modelling purposes with a focus on optimising data pipelines Identify streamline and optimise inefficient processes Prepare and deliver presentations and visual materials that effectively communicate software design decisions and model logic to … first degree or higher degree in a highly numerate subject is essential Minimum 2 years' experience in Python development, including scientific computing and data science libraries (NumPy, pandas, SciPy, PySpark) Solid understanding of object-oriented software engineering design principles for usability, maintainability and extensibility Experience working with Git More ❯

As our next Technical Engineer, you'll join our Leeds Based Data Technology Solutions function to shape, lead, coordinate, and transition technical change into service. This role offers hybrid working with working from home capability. What You'll Do Identify, shape, scope and lead development work. Create delivery … of design activities Primary contact for IT, solution architecture, product owners, infrastructure, testing and release. Work with colleagues in these areas to review data solution options to assess impact of new requirements on existing functionality, data and systems including alternative design scenarios. Perform detailed gap analysis … ll Do It Expert knowledge of current technologies and platforms. Examples are Oracle (Database, ODI, Pl-SQL), SAP Business Objects, SAP business objects data services, Denodo Virtualisation, MDM solutions, reference data management Experience in Meta data driven design for Data pipelines Proven More ❯

We are seeking an experienced Kafka Real-Time Architect to design and implement scalable, high-performance real-time data processing systems leveraging Apache Kafka. In this role, you will be responsible for architecting and managing Kafka clusters, ensuring system scalability and availability, and integrating Kafka with various data processing platforms. This position requires a deep understanding of Kafka and related technologies, along with a proactive approach to addressing business data needs and ensuring optimal system performance. Key Responsibilities: Design & Architecture: Architect and design scalable, real-time streaming systems using Apache Kafka, ensuring they are … robust, highly available, and meet business requirements for data ingestion, processing, and real-time analytics. Kafka Cluster Management: Configure, deploy, and maintain highly available Kafka clusters, ensuring scalability and fault tolerance across distributed environments. Manage configurations, optimize cluster performance, and troubleshoot issues to maintain smooth operations. Integration & Data More ❯