1 to 25 of 27 Data Pipeline Jobs in West Yorkshire

Purpose of the Job: Design, build, and maintain robust data systems and pipelines that support data storage, processing, and analysis on the Cloud. Work with large datasets, ensuring data quality, scalability, and performance, while collaborating. closely with data scientists, analysts, and other engineering teams to understand their data needs and … provide them with high-quality, accessible data. They are responsible for ensuring that the underlying data infrastructure supports the organizations broader data and business goals, enabling more effective data-driven decision-making. Key Accountabilities: • Design and implement scalable, efficient, and secure data architectures, ensuring optimal data flow across systems in … and process across the team along with BI lead. • Influence the evolution of business and system requirements and contribute to the design of technical solutions to feed a delivery pipeline that increasingly employs Agile methods such as SCRUM and Kanban • You will be required to develop unit tested code and then support test cycles including post implementation validation. You More ❯

Location: Worcester Duration: 6 month initial contract Rate: (Outside IR35) Security: Active DV clearance required Role details: We are looking for 3 x Data Engineers to join our defence & security client on a contract basis. You will helpdesign, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi … These roles are supporting our clients team in Worcester (fully onsite), and requires active UK DV clearance. Key Responsibilities: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi Implement data ingestion, transformation, and integration processes, ensuring data quality and security Collaborate with data … cross-functional teams. Detail-oriented with a focus on ensuring data accuracy, quality, and security. Proactive problem-solving mindset and ability to troubleshoot complex data pipeline issues Desirable skills: Experience working in government, defence, or highly regulated industries with knowledge of relevant standards. Experience with additional data processing and ETL tools like Apache More ❯

Location: Worcester Duration: 6 month initial contract Rate: (Outside IR35) Security: Active DV clearance required Role details: We are looking for 3 x Data Engineers to join our defence & security client on a contract basis. You will helpdesign, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi … These roles are supporting our clients team in Worcester (fully onsite), and requires active UK DV clearance. Key Responsibilities: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi Implement data ingestion, transformation, and integration processes, ensuring data quality and security Collaborate with data … cross-functional teams. Detail-oriented with a focus on ensuring data accuracy, quality, and security. Proactive problem-solving mindset and ability to troubleshoot complex data pipeline issues Desirable skills: Experience working in government, defence, or highly regulated industries with knowledge of relevant standards. Experience with additional data processing and ETL tools like Apache More ❯

Location: Worcester Duration: 6 month initial contract Rate: (Outside IR35) Security: Active DV clearance required Role details: We are looking for 3 x Data Engineers to join our defence & security client on a contract basis. You will helpdesign, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi … These roles are supporting our clients team in Worcester (fully onsite), and requires active UK DV clearance. Key Responsibilities: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi Implement data ingestion, transformation, and integration processes, ensuring data quality and security Collaborate with data … cross-functional teams. Detail-oriented with a focus on ensuring data accuracy, quality, and security. Proactive problem-solving mindset and ability to troubleshoot complex data pipeline issues Desirable skills: Experience working in government, defence, or highly regulated industries with knowledge of relevant standards. Experience with additional data processing and ETL tools like Apache More ❯

a range of benefits to support your finances, wellbeing and family. Working Pattern: Permanent Full Time Flexible First options available The Role We're looking for a Senior Data Engineer to join the Asset Data & Insights value stream. You'll lead the design, development and operations of robust, scalable data pipelines using modern cloud … technologies-empowering real-time analytics, advanced BI, and data-driven decision-making across SSE Thermal. You'll need hands-on expertise with the Databricks ecosystem, a strong command of data engineering best practices, and a growth mindset that balances technical excellence with collaboration and continuous improvement. Experience in an asset-intensive or energy environment is desirable … Delta Live Tables, Unity Catalog) to serve downstream analytics and reporting. - Optimise data architecture for performance, scalability, and reliability, proactively monitoring data health and troubleshooting pipeline issues. - Mentor junior engineers and foster a knowledge-sharing culture through documentation, pairing, and architectural guidance. - Collaborate with analysts, product teams, and business stakeholders to translate requirements into performant More ❯

Data Engineering Manager Location: Leeds (Mostly Remote – 1 Day On-Site) Salary: Up to £75,000 per annum Are you an experienced Senior Data Engineer ready to step into a leadership role? We’re looking for a Lead Data Engineer to join our team based in Leeds, working mostly remotely with just one day … on-site per week. You’ll lead the design and delivery of scalable, cloud-based data solutions using Databricks, Python, and SQL, while mentoring a team and driving engineering best practices. About You You might currently be a Senior Data Engineer ready to grow your leadership skills. You’re passionate about building robust, efficient data pipelines and shaping cloud data architecture in an agile environment. Key Responsibilities Lead development of data pipelines and solutions using Databricks, Python, and SQL Design and maintain data models supporting analytics and business intelligence Build and optimise ELT/ETL processes on AWS or Azure Collaborate closely with analysts, architects, and stakeholders More ❯

Senior Data Engineer My client is looking for a Senior Data Engineer to join their expanding data team. This is an excellent opportunity for someone who thrives in a collaborative, insight-driven environment and is passionate about leveraging data to deliver tangible business value. This is not a role for someone just … moving data from A to B with Python its for someone who understands how to model, optimise, and apply data to support better business decisions. Key Experience Required: Expertise in SQL Server (approx. 75% of the role) including performance tuning, data warehousing, and reporting. Experience with AWS data services (approx. 25%), particularly … RDS, S3, Lambda, EventBridge, and building data pipelines in cloud-native environments. Proven delivery of dimensional/Kimball data models in collaboration with Data Analysts. Strong proficiency with Power BI, including performance optimisation and deployment strategies. Hands-on experience with Python for automation and data transformation (not just file handling). Understanding More ❯

JOB TITLE: Senior Data Engineer SALARY: £70,929 - £78,810 LOCATION: Edinburgh, Manchester, Leeds or Bristol HOURS: Full time WORKING PATTERN: Hybrid, 40% (or two days) in an office site About this opportunity At Lloyds Banking Group, we're moving to the next stages of our digital transformation, and it's our job within the Chief Security Office … CSO) to ensure that we keep our customers, colleagues and assets safe from threat. The CSO's Security Data Services lab does this by using data to build security insights by working collaboratively with our Data Management and Analytics teams. This is an exciting opportunity for a Senior Data Engineer to join … the Security Data Services lab to collaborate with the Data Operations, Software Engineers, Data Scientists, and Business SMEs to design, develop, integrate and test Data Product features on the product roadmaps to meet customer needs. What you'll do Engineer Data Products that are stable, scalable, performant, accessible, testable and More ❯

Senior Data Engineer Hybrid 2x per week (Leeds, West Yorkshire) Salary to £60,000 + benefits About the Role: Our client, a tech for good organisation, is investing heavily in digital transformation and building a cloud-first, data-led approach to support major environmental goals. Think scalable pipelines, secure platforms, and data systems that … directly contribute to a greener future. As a Senior Data Engineer, you'll play a key role in designing and delivering robust, cloud-based data infrastructure. You'll work with large datasets, build scalable pipelines, and help ensure data is high quality, secure, and accessible, while mentoring more junior engineers and working closely with … stakeholders across the business. Key Responsibilities: Develop scalable and secure data architectures ensuring optimal data flow Cloud data platform development, data modelling, shaping and technical planning Implementing best practices around data security, governance, privacy, and compliance Ensure company data is secure, accurate, maintained and available. Mentor data More ❯

About The Role Hippo is recruiting for a Principal Data Engineer to join our Hippo Herd. Principal Data Engineers work in multi-disciplinary teams that build, support & maintain User-Centred digital solutions that offer real value and work for everyone. Hippo's Principal Data Engineers are experts in their field. You will act as … a principal consultant providing technical expertise in data engineering practices both internally and to our clients. You will drive the team's technical deliverables, maintain client relationships and be passionate about developing and upskilling others. Please note, we are looking for candidates who are looking for growth at this level (principal), therefore the advertised salary band is the … team and stakeholders, identifying and ensuring that challenges are overcome Lead in the recruitment of other engineers, and support other consultants in their professional development Ensure Hippo's Data and Engineering communities contribute to Hippo's business at a strategic level Lead on new business opportunities, writing technical bids and proposals Promote Hippo's Data Herd More ❯

days in office Leeds £60K We are looking for a BI Developer to join our clients team in Leeds. The ideal candidate will have experience in PowerBI, Data Engineering, and have a strong understanding of data warehousing and business intelligence. The role will involve working with a team of data engineers to design, develop … and maintain data pipelines. You will also be responsible for creating and managing data models, and developing reports and dashboards using PowerBI. The successful candidate will be a highly motivated and self-directed individual with excellent communication and teamwork skills. They will also have a strong understanding of data analysis and visualization techniques. Responsibilities … Create and manage data models Designing, developing, testing, and implementing robust data applications powered by cutting-edge technologies like Azure DataFactory, PowerBI, Qlikview, and SSRS. Develop reports and dashboards using PowerBI Work with a team of data engineers to improve data quality and performance Provide technical support to users of PowerBI Qualifications More ❯

AWS Data Engineer - Manchester - Hybrid We're working with a global insurance company that's redefining the industry through technology. As a key end user of AWS, they're leveraging cloud-native solutions and advanced analytics to transform how they serve millions of customers. They're looking for Data Engineers who are passionate about building scalable … data platforms and want to be part of a forward-thinking team. You'll work on modern data pipelines, cloud infrastructure, and real-time analytics projects that directly impact business outcomes. As an AWS Data Engineer, you'll play a critical role in building and maintaining our cloud-based data infrastructure. You … ll work closely with data scientists, analysts, and other engineers to design, develop, and deploy scalable data solutions on the AWS platform. The client operates with a hybrid working model, with two days per week in their Manchester office. Responsibilities: -Design, build, and maintain data pipelines using AWS services such as S3, Glue, Lambda More ❯

Location: Leeds City Centre + Hybrid Working Job Type: Full-time Team: Product Reports to: Head of Product Development About Us: We use data science and AI to help retailers make profitable pricing, marketing and retail media decisions. Put simply, we help retailers sell the right products, to the right people, for the right price. We offer a … 2+ years of experience in a business intelligence development or related role Excellent SQL skills Good understanding of relational and analytical database architectures and methodologies Experience of cloud data warehousing solutions e.g. Snowflake, BigQuery, Redshift Good hands-on experience with at least one Business Intelligence platform Strong analytical and problem-solving skills Excellent communication skills, with the ability … to explain technical details to non-technical stakeholders Comfortable with occasional client interaction in a professional and supportive capacity Great UX & data visualisation skills The Role: Design, develop and maintain BI solutions including dashboards, reports and data models Produce visually engaging deliverables for consumption by retail clients Write complex SQL queries and scripts for data More ❯

Role: Data Engineering Manager Salary: £65,000- £75,000 per annum Location: Leeds (One day on site). VIQU have partnered with a leading supply chain organisation who are looking to expand their data teams. The Data Engineering Manager will manage a team of eight to help with an on-going digital transformation. The … the clients Databricks platform (AWS is preferred but open to Azure.GCP experience also), utilising Python and SQL, contribute to CI/CD pipelines, strategy development, cost optimisation and data governance frameworks. Job duties of the Data Engineering Manager: Manage a team of eight engineers, helping to mentor and coach the team. Manage the adoption of automated … experience in a managerial role, coaching similar sized teams. Deep knowledge of the Databricks platform. Hands on Python development experience. SQL optimisation. Experience with large scale data pipeline optimisation. Experience with Streaming and Batch Spark workloads. Strong people management skills. Role: Data Engineering Manager Salary: £65,000- £75,000 per annum Location: Leeds (One day More ❯

Role: Data Engineering Manager Salary: £65,000- £75,000 per annum Location: Leeds (One day on site). VIQU have partnered with a leading supply chain organisation who are looking to expand their data teams. The Data Engineering Manager will manage a team of eight to help with an on-going digital transformation. The … the clients Databricks platform (AWS is preferred but open to Azure.GCP experience also), utilising Python and SQL, contribute to CI/CD pipelines, strategy development, cost optimisation and data governance frameworks. Job duties of the Data Engineering Manager: Manage a team of eight engineers, helping to mentor and coach the team. Manage the adoption of automated … experience in a managerial role, coaching similar sized teams. Deep knowledge of the Databricks platform. Hands on Python development experience. SQL optimisation. Experience with large scale data pipeline optimisation. Experience with Streaming and Batch Spark workloads. Strong people management skills. Role: Data Engineering Manager Salary: £65,000- £75,000 per annum Location: Leeds (One day More ❯

AWS Data Engineer - IT Consultancy - Leeds - Hybrid Are you a data engineer with a passion for AWS and solving real-world business problems? I am working with a fast-growing IT consultancy based in Leeds that's delivering cutting-edge cloud and data solutions to clients across the UK. They're looking for their … next AWS Data Engineer to join the team. As an AWS Data Engineer, you'll be working on a variety of client projects, helping to design, build, and optimise scalable data pipelines and cloud-native architectures. You'll collaborate with cross-functional teams to deliver high-impact solutions that drive business value. Requirements: Experience … as a Data Engineer using AWS: Glue, Lambda, S3, Redshift, Athena, EMR. Snowflake experience (Not essential) Strong Python SQL & NoSQL databases CI/CD tools : GitHub Actions or CodePipeline Infrastructure as Code (Terraform is preferable) Experience building ETL/ELT pipelines Familiarity with data lake and data warehouse concepts Excellent communication and stakeholder engagement More ❯

Data Engineer – Leeds– SQL | Power BI | Azure | ETL Ready to drive innovation with T-SQL and BI, shaping a cutting-edge data platform? Do you have the expertise to build and maintain complex data warehouses and Power BI solutions? Excited to solve BI challenges and support data-driven decision-making across the … business? A fantastic new opportunity has arisen with a professional services company for an experienced Data Engineer. You'll be joining a growing team with offices across the UK, committed to ambitious growth and employee wellbeing. This role is based in their Leeds office, offering hybrid working. What's in it for you? Salary up to … Factory to ensure the warehouse can support large-scale reporting and analytics needs. Your T-SQL expertise will be crucial for managing complex queries and troubleshooting data pipeline issues. Key Skills: SQL/T-SQL Power BI or Tableau Experience with data warehousing and ETL/ELT processes Azure If you're passionate about shaping More ❯

a Manual QA Test Engineer with hands-on experience in SnapLogic to join my clients growing team on a contract basis. You will be responsible for testing integrations, data pipelines, and workflows built using the SnapLogic platform, ensuring quality, stability, and performance across various projects. Key Responsibilities Design and execute manual test cases for SnapLogic pipelines and integrations. … Validate data transformations, API integrations, and ETL workflows. Collaborate with developers, business analysts, and integration architects to understand functional and non-functional requirements. Identify, document, and track bugs through to resolution. Maintain test documentation, including test cases, test results, and issue logs. Support UAT and production release verification. Ensure data accuracy and integrity across systems and … integrations. Required Skills & Experience 1+ years of experience testing SnapLogic integrations (required). Strong background in manual testing , particularly for APIs, data pipelines, and enterprise integrations. Experience with JSON, REST APIs, SQL , and data validation. Familiarity with tools like Postman, JIRA, Confluence, or similar. Strong analytical and problem-solving skills. Excellent communication skills and attention to More ❯

Lead Data Engineer - Remote - UK About the Role: We're hiring on behalf of a fast-growing, tech-driven organisation looking for a Lead Data Engineer to take ownership of their data infrastructure. This is a fully remote role open to candidates based anywhere in the UK. You'll lead the design and development … of scalable data pipelines, mentor a small team of engineers, and work closely with cross-functional teams to deliver high-impact data solutions. Key Responsibilities: Architect and maintain robust data pipelines using AWS services (Glue, Lambda, S3, Airflow) Lead the migration and optimisation of data workflows into Snowflake. Collaborate with analysts, data scientists, and product teams to deliver clean, reliable data. Define and enforce best practices in data engineering and governance. Mentor junior engineers and contribute to a culture of technical excellence Requirements: Proven experience in a Lead or Senior Data Engineering role Strong hands-on expertise with AWS (Glue, Lambda, S3) Deep experience with Snowflake More ❯

Data Quality and Governance Manager - Hybrid - Leeds - £65,000 My client is looking for a Data Quality & Governance Manager to shape how data is trusted and used throughout their organisation. They are looking for someone to lead efforts in ensuring the consistency, reliability, and governance of data. This role will work closely with business … and technical teams to develop and implement frameworks for data quality, governance, and master data management, using Snowflake and Microsoft Azure. Requirements: Experience in data governance and data quality management Strong knowledge of data warehouse architecture and cloud platforms, especially Snowflake and Microsoft Azure. Proven experience designing and managing data pipelines in cloud environments with integrated data governance principles. Hands-on experience with data migrations Strong experience in working with data engineers, business stakeholders, and data stewards. Please Note: This is role for UK residents only. This role does not offer Sponsorship. You must have the right to work in More ❯

Data Quality and Governance Manager - Hybrid - Leeds - £65,000 My client is looking for a Data Quality & Governance Manager to shape how data is trusted and used throughout their organisation. They are looking for someone to lead efforts in ensuring the consistency, reliability, and governance of data. This role will work closely with business … and technical teams to develop and implement frameworks for data quality, governance, and master data management, using Snowflake and Microsoft Azure. Requirements: Experience in data governance and data quality management Strong knowledge of data warehouse architecture and cloud platforms, especially Snowflake and Microsoft Azure. Proven experience designing and managing data pipelines in cloud environments with integrated data governance principles. Hands-on experience with data migrations Strong experience in working with data engineers, business stakeholders, and data stewards. Please Note: This is role for UK residents only. This role does not offer Sponsorship. You must have the right to work in More ❯

Data Architect £80 - £95k + 15% Bonus Hybrid - Leeds 2 days per month Excellent benefits Our client is a privately owned, global insurance services provider seeking an experienced Data Architect to join them at a pivotal moment, as they embark on an exciting transformation program aimed at optimising operations and enhancing the customer experience. They're … looking for an experienced Data Architect to lead the design, development, and governance of their data architecture - aligning it with business goals, improving services, boosting operational efficiency, and shaping the organisation's future data strategy. Technology: While they operate primarily within a Microsoft environment, they welcome candidates with experience in other technologies. What matters … most is finding the right person to lead change, inspire innovation, and collaborate closely with a talented team of Data Engineers and Analysts. Key responsibilities: Architect end-to-end data solutions using Azure Build secure, scalable data pipelines and lakehouse architectures Collaborate with data engineers, analysts, and business teams Champion data More ❯

Head of Data - Manchester - Hybrid Are you ready to shape the future of data in retail? We're working exclusively with a leading name in the retail sector who are investing heavily in their data capabilities. As part of this transformation, they're hiring their first-ever Head of Data - a pivotal … role that will define and drive the data strategy across the business. The Role: This is a brand-new leadership position, reporting directly into the C-suite. You'll be responsible for building and leading a small but growing team of data professionals, while also being hands-on in shaping the data architecture and … strategy. You'll work closely with stakeholders across the business to unlock the power of data, enabling smarter decision-making and driving innovation. What You'll Be Doing: Define and execute the company's data strategy from the ground up Lead and grow a high-performing data team Oversee the design and implementation of More ❯

Snowflake Data Engineer - £60,000-£80,000 My client is looking for a skilled Data Engineer with expertise in Snowflake to join their team. You will build and manage data pipelines, ensure integration of data into Snowflake and implement data government strategies. The company has had substantial expansion and received … great opportunity to work in a fast-paced, technology driven business and work alongside experienced colleagues at all different levels. Requirements: -Snowflake -Cloud-based platform experience -Experience building data pipelines in Snowflake -Stakeholder management Please Note: This is role for UK residents only. This role does not offer Sponsorship. You must have the right to work in the More ❯

Snowflake Data Engineer - £60,000-£80,000 My client is looking for a skilled Data Engineer with expertise in Snowflake to join their team. You will build and manage data pipelines, ensure integration of data into Snowflake and implement data government strategies. The company has had substantial expansion and received … great opportunity to work in a fast-paced, technology driven business and work alongside experienced colleagues at all different levels. Requirements: -Snowflake -Cloud-based platform experience -Experience building data pipelines in Snowflake -Stakeholder management Please Note: This is role for UK residents only. This role does not offer Sponsorship. You must have the right to work in the More ❯