1 to 25 of 32 Data Pipeline Jobs in Yorkshire

allowance based on a skills assessment Published on 10 July 2025 Deadline 30 July 2025 Location Croydon, Manchester, Sheffield About the job Job summary As a Lead Data Architect, you will create and uphold data architecture across critical Home Office systems, enabling secure, data-driven services that keep the country safe. Working under the … vision set by the Principal Data Architect, you'll design robust data models, metadata systems, and governance frameworks to serve diverse stakeholders. Your leadership will span strategic planning, mentoring junior data architects, and aligning technical decisions with business objectives. If you thrive on architecting solutions that integrate emerging technologies such as cloud-native platforms … data lakehouses, streaming data pipelines and AI-powered analytics while collaborating with multi disciplinary teams, this role offers you the chance to elevate how the Home Office exploits data for maximum public impact. Due to business requirements this post is available on a full time/flexible working basis. Where business needs allow, some More ❯

Senior Data Ops Data EngineerLeeds 2/3 days per week£55,000-£67,000+ Excellent Benefits18 Months FTC with long term potential Your New Company Our large public sector organisation are seeking a senior Data Platform DataOps Engineer to serve as our clients first DataOps specialist in a growing team of Data Engineers and DevOps professionals.In this pivotal role, you will focus on operationalising and automating their data lifecycle to ensure that data workflows perform with reliability and efficiency. You will integrate CI/CD data pipelines, streamline deployment processes, enforce robust data governance, and optimise operational costs within our Microsoft Azure … APIM. Proficiency in developing CI/CD data pipelines and strong programming skills in Python, SQL, Bash, and PySpark for automation. Strong aptitude for data pipeline monitoring and an understanding of data security practices such as RBAC and encryption. Implemented data and pipeline observability dashboards, ensuring high data More ❯

Job summary Ready to take on a career-defining challenge? HDFT are looking for two Senior Data Engineers to play a pivotal role in a ground-up deployment of Microsoft Fabric. This is a rare opportunity to work on a completely greenfield project, shaping architecture and developing robust, scalable data pipelines using best-fit technologies. What … s in it for you? - Immediate exposure to Microsoft Fabric -- be at the forefront of cutting-edge data engineering. - Greenfield deployment -- minimal legacy constraints, just innovation. - Critical role -- influence architecture, tooling, and best practices from day one. - Technology-first environment -- work with modern cloud and data technologies to build something that lasts. What you'll be … doing - Designing and implementing high-performance data pipelines for real time and batch processing. - Architecting a future-proof data platform, leveraging Microsoft Fabric and other best-in-class tools. - Collaborating with cross-functional teams to drive data strategy and governance.- Optimising performance, ensuring scalability, security, and reliability. What we're looking for - Proven experience More ❯

Purpose of the Job: Design, build, and maintain robust data systems and pipelines that support data storage, processing, and analysis on the Cloud. Work with large datasets, ensuring data quality, scalability, and performance, while collaborating. closely with data scientists, analysts, and other engineering teams to understand their data needs and … provide them with high-quality, accessible data. They are responsible for ensuring that the underlying data infrastructure supports the organizations broader data and business goals, enabling more effective data-driven decision-making. Key Accountabilities: • Design and implement scalable, efficient, and secure data architectures, ensuring optimal data flow across systems in … and process across the team along with BI lead. • Influence the evolution of business and system requirements and contribute to the design of technical solutions to feed a delivery pipeline that increasingly employs Agile methods such as SCRUM and Kanban • You will be required to develop unit tested code and then support test cycles including post implementation validation. You More ❯

and want to take your career further in a thriving office environment? Are you a strong communicator who works well within a team? Do you understand data pipeline tools and data modelling? Were seeking an SQL Developer to join our expanding Database team in Leeds! In the role, you will: Analyse business requirements through collaboration … with stakeholders, IT Management, and the user community Develop new systems, integrate third-party platforms, and maintain existing applications Use data to deliver business improvements and support informed decision-making Work with stakeholders on data extraction and manipulation tasks Write and optimise in-application SQL statements Provide user support for report errors, debugging, and issue resolution … experience in a database developer/data engineer role Proficiency in SQL and Python Experience with cloud platforms (AWS, GCP, or Azure) Familiarity with data pipeline tools (e.g., SSIS, Spark) Strong grasp of data warehousing and data modelling principles Experience undertaking development work in C# Ability to create and troubleshoot SSRS More ❯

Senior Data Engineer Hybrid 2x per week (Leeds, West Yorkshire) Salary to £60,000 + benefits About the Role: Our client, a tech for good organisation, is investing heavily in digital transformation and building a cloud-first, data-led approach to support major environmental goals. Think scalable pipelines, secure platforms, and data systems that … directly contribute to a greener future. As a Senior Data Engineer, you'll play a key role in designing and delivering robust, cloud-based data infrastructure. You'll work with large datasets, build scalable pipelines, and help ensure data is high quality, secure, and accessible, while mentoring more junior engineers and working closely with … stakeholders across the business. Key Responsibilities: Develop scalable and secure data architectures ensuring optimal data flow Cloud data platform development, data modelling, shaping and technical planning Implementing best practices around data security, governance, privacy, and compliance Ensure company data is secure, accurate, maintained and available. Mentor data More ❯

a great place to work and as our business expands, we are looking for ambitious, talented people to join us. We are looking for a technically proficient Senior Data Engineer to join our growing Data team. Your primary focus will be on ensuring data quality, stability, and reliability … from the moment data arrives in its rawest form to when it is used in decision-making dashboards and customer-facing reports. You will optimise the transformation pipeline from start to finish, guaranteeing that datasets are robust, tested, secure, and business-ready. Our data platform is built using Databricks, with data pipelines written … in PySpark and orchestrated using Airflow. You will be expected to challenge and improve current transformations, ensuring they meet our performance, scalability, and data governance needs. This includes work with complex, nested data structures, ensuring they are reliably parsed and transformed. Experience in managing sensitive data (PII) and implementing GDPR policies is required. You More ❯

Job Position: Graduate Data Analyst (Engineer/Scientist) - via the Graduate Development Programme Locations: Sheffield and South Yorkshire Starting Salary: £25,000 Application Requirements: Minimum 2:1 or above in a STEM (Science, Technology, Engineering, and Math) subject. Experience in analysing data. Right to work in the UK unsponsored for the duration of the programme. Ability to work … delivering excellence. If you have a keen attention to detail and accuracy, a knack for spotting trends and patterns and the ability to tell a compelling story using data, you'll thrive in our environment. We value resourcefulness, resilience, and a passion for driving change. Join us and be part of a community that cares, collaborates, and thrives … together. Understanding the types of roles available to a Data Analyst: Data Analyst: Kick-start your career as a Data Analyst, where you'll transform complex data into actionable insights, create dynamic visualisations and drive business decisions. You'll work with cutting-edge tools, collaborate with stakeholders, and develop automated reports and More ❯

Our Senior Data Engineers enable public sector organisations to embrace a data-driven approach by providing high-quality, cost-efficient data platforms and services tailored to clients' needs. They develop, operate, and maintain these services, ensuring maximum value for data consumers, including analysts, scientists, and business stakeholders. Key responsibilities As a Senior … Data Engineer, you may assume multiple roles based on our clients' needs. The role is highly hands-on, supporting project delivery as a senior contributor and upskilling client team members. You might also take on a technical architect role, collaborating with the MadeTech team to identify growth opportunities within the account. You'll need a drive to deliver … criteria. Enthusiasm for learning and self-development Proficiency in Git (including Github Actions) and understanding of branch strategies Experience gathering and meeting requirements from clients and users on data projects Strong experience in Infrastructure as Code (IaC) and deploying infrastructure across environments Managing cloud infrastructure with a DevOps approach Handling and transforming various data types (JSON More ❯

SENIOR ANALYTICS ENGINEER UP TO £80,000 + BENEFITS PRIMARILY REMOTE Looking to take ownership of scalable data models, shape a brand-new analytics function, and work with the modern data stack from day one? THE COMPANY: I'm working with a fast-scaling, tech-driven organisation that's investing heavily in its data capabilities. As part of a wider data transformation, they're building out a dedicated Analytics Engineering team focused on creating scalable, trusted data infrastructure. With strong support from BI, Data Science, and Engineering teams, this function is central to how the business defines and uses data going forward. THE ROLE … skills and hands-on experience with dbt (or similar tools) Experience designing as well as building data models Exposure to Snowflake and/or data pipeline tools Understanding of testing, CI/CD, and data quality frameworks THE BENEFITS: You will receive a salary dependent on experience-up to £80,000. On top More ❯

now to be part of our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of roles including but not … limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for further analysis and investigation. Working with others is key, you could be working with other engineers, developers More ❯

now to be part of our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety of roles including but not … limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the data is organized and available for further analysis and investigation. Working with others is key, you could be working with other engineers, developers More ❯

Role: Data Engineering Manager Salary: £65,000- £75,000 per annum Location: Leeds (One day on site). VIQU have partnered with a leading supply chain organisation who are looking to expand their data teams. The Data Engineering Manager will manage a team of eight to help with an on-going digital transformation. The … the clients Databricks platform (AWS is preferred but open to Azure.GCP experience also), utilising Python and SQL, contribute to CI/CD pipelines, strategy development, cost optimisation and data governance frameworks. Job duties of the Data Engineering Manager: Manage a team of eight engineers, helping to mentor and coach the team. Manage the adoption of automated … experience in a managerial role, coaching similar sized teams. Deep knowledge of the Databricks platform. Hands on Python development experience. SQL optimisation. Experience with large scale data pipeline optimisation. Experience with Streaming and Batch Spark workloads. Strong people management skills. Role: Data Engineering Manager Salary: £65,000- £75,000 per annum Location: Leeds (One day More ❯

days in office Leeds £60K We are looking for a BI Developer to join our clients team in Leeds. The ideal candidate will have experience in PowerBI, Data Engineering, and have a strong understanding of data warehousing and business intelligence. The role will involve working with a team of data engineers to design, develop … and maintain data pipelines. You will also be responsible for creating and managing data models, and developing reports and dashboards using PowerBI. The successful candidate will be a highly motivated and self-directed individual with excellent communication and teamwork skills. They will also have a strong understanding of data analysis and visualization techniques. Responsibilities … Create and manage data models Designing, developing, testing, and implementing robust data applications powered by cutting-edge technologies like Azure DataFactory, PowerBI, Qlikview, and SSRS. Develop reports and dashboards using PowerBI Work with a team of data engineers to improve data quality and performance Provide technical support to users of PowerBI Qualifications More ❯

Lead Data Architect £67,000 - 76,000 GBP Hybrid WORKING Location: Leeds, Yorkshire and the Humber - United Kingdom Type: Permanent Lead Data Architect Locations: Leeds, Manchester, Birmingham, Bristol, Glasgow, or London (Hybrid - up to 2 days on-site per week, remainder remote) Salary: £67,000 - £76,000 (non-London)/Up to £83,000 (London) + … Benefits A respected digital consultancy is hiring a Lead Data Architect to join its growing Data Engineering practice. Known for delivering modern, human-centred services across public and private sectors, this organisation supports clients through meaningful transformation by combining data, design, and engineering. This role is ideal for experienced architects who enjoy shaping cloud … native data platforms, working in multidisciplinary teams, and embedding data governance and architectural rigour across complex service ecosystems. The Role: As a Lead Data Architect, you'll play a key role in designing scalable, secure, and accessible data platforms that enable insight-driven decision-making. You'll work closely with engineering teams More ❯

Lead Data Engineer - Leeds This is an exciting opportunity for a skilled Lead Data Engineer to advance their career while remaining deeply technical. The role offers a clear path for progression without requiring a move into management or sales, allowing individuals to focus on what they do best-engineering high-quality, scalable data solutions. … ensure that individuals are recognised and supported-especially during demanding projects. The business is committed to employee wellbeing and fosters a positive, inclusive culture. Role Overview The Lead Data Engineer will take a hands-on technical leadership role, guiding the delivery of high-impact solutions within cloud-based environments. This position is ideal for professionals who are passionate … about modern data engineering and want to influence best practices, architectural decisions, and agile delivery across teams. The role involves close collaboration with multidisciplinary teams and requires strong communication skills to translate technical approaches for non-technical audiences, including senior stakeholders and clients. Key Responsibilities Lead the design, development, and optimisation of robust data pipelines and More ❯

Lead Data Engineer ££64,000 GBP Hybrid WORKING Location: Leeds, Yorkshire and the Humber - United Kingdom Type: Permanent This is an exciting opportunity for a skilled Lead Data Engineer to advance their career while remaining deeply technical. The role offers a clear path for progression without requiring a move into management or sales, allowing individuals to … focus on what they do best-engineering high-quality, scalable data solutions. Why Join? This organisation values continuous learning and personal growth. Team members benefit from a dedicated learning allowance, enabling them to shape their own development journey. Paid overtime and a strong emphasis on work-life balance ensure that individuals are recognised and supported-especially during demanding … projects. The business is committed to employee wellbeing and fosters a positive, inclusive culture. Role Overview The Lead Data Engineer will take a hands-on technical leadership role, guiding the delivery of high-impact solutions within cloud-based environments. This position is ideal for professionals who are passionate about modern data engineering and want to influence More ❯

Azure Data Engineer - Leeds (Hybrid) - £65,000 Are you an experienced Azure Data Engineer with a passion for building scalable, cloud-native data solutions? We're recruiting on behalf of a dynamic organisation in Leeds that's investing in its data platform - and they're looking for someone with strong Databricks expertise … to join their team. About the role: Designing and developing robust data pipelines using Azure Data Factory, Databricks, and Synapse Analytics. Working with Delta Lake and Azure Data Lake Storage to manage and optimise large datasets. Collaborating with data analysts, engineers, and business stakeholders to deliver clean, reliable data. Supporting the migration … of legacy systems to a modern Azure-based architecture Ensuring best practices in data governance, security, and performance tuning Requirements: Proven experience with Azure Data Services (ADF, Synapse, Data Lake) Strong hands-on experience with Databricks (including PySpark or SQL) Solid SQL skills and understanding of data modelling and ETL/ELT More ❯

Job Title: Senior Data Engineer Salary: Up to £60,000 Location: Leeds (Hybrid - 2 days a week in the office) Are you a data engineering expert ready to take the next step in your career? We're hiring a Senior Data Engineer to join our growing team in Leeds. You'll play a critical … role in designing, building, and managing modern data pipelines and cloud-based data solutions. What you'll be doing: Designing and maintaining scalable data pipelines Building and optimising data warehousing solutions Working with Databricks … and modern cloud platforms (Azure or AWS) Collaborating with cross-functional teams to deliver high-impact data products Leading best practices in data engineering and pipeline architecture What we're looking for: Proven experience in data engineering at a senior level Strong hands-on knowledge of Databricks Experience with Azure or AWS cloud More ❯

At Amazon, data is critical to everything we do, from understanding product inventories across our global fulfillment centers to predicting what products to recommend to customers. Data ceaselessly informs our flywheel, helping us to understand customer needs, onboard new sellers and selection, and identify opportunities for cost savings. Business Data Technologies (BDT) provides trustworthy … intuitive, and cost-efficient solutions for Amazonians to secure, store, analyze, and transform this data at unparalleled scale. BDT owns, develops, and operates Andes, one of the largest data lakes in the world, storing more than 2 exabytes of data, read/written to by 100k+ data pipelines, with >60k monthly active … users. To do this, we need extraordinary individuals to build the next generation of data analytics, machine learning, and AI solutions for thousands of scientists, engineers, and analysts who rely upon us to get their jobs done in a safe and secure manner. Data Privacy and Comprehension is a key function of an enterprise datalake. The More ❯

of Engineering. You will oversee all aspects of the engineering lifecycle, from technology design and architecture through to implementation and optimisation. You'll work closely with cloud architects, data scientists, and risk officers to deliver a robust, scalable, and compliant platform that meets the bank's risk and regulatory needs. The role will shape the Lab vision, set … and demonstrate technical leadership across the lifecycle from idea to value realisation. About us Like the modern Britain we serve, we're evolving. Investing billions in our people, data and tech to transform how we meet the changing needs of our 26 million customers. We're growing with purpose. Join us on our journey and you will too … What you'll do Lead the engineering team responsible for end-to-end infrastructure and technology transformation, including legacy decommissioning and GCP-based data infrastructure. Provide technical direction and mentorship, fostering collaboration, innovation, and continuous improvement. Design scalable, high-performance cloud infrastructure on GCP, ensuring availability, resilience, security, and compliance. Implement Infrastructure-as-Code (IaC) using tools like More ❯

Lead Data Engineer - Remote - UK About the Role: We're hiring on behalf of a fast-growing, tech-driven organisation looking for a Lead Data Engineer to take ownership of their data infrastructure. This is a fully remote role open to candidates based anywhere in the UK. You'll lead the design and development … of scalable data pipelines, mentor a small team of engineers, and work closely with cross-functional teams to deliver high-impact data solutions. Key Responsibilities: Architect and maintain robust data pipelines using AWS services (Glue, Lambda, S3, Airflow) Lead the migration and optimisation of data workflows into Snowflake. Collaborate with analysts, data scientists, and product teams to deliver clean, reliable data. Define and enforce best practices in data engineering and governance. Mentor junior engineers and contribute to a culture of technical excellence Requirements: Proven experience in a Lead or Senior Data Engineering role Strong hands-on expertise with AWS (Glue, Lambda, S3) Deep experience with Snowflake More ❯

The Data Analyst will be responsible for collecting, processing, and analysing data to support strategic decision-making & planning across the business. Role Analyse large datasets to identify trends, patterns, and insights Develop and maintain dashboards and reports using tools like Power BI Perform data cleaning, transformation, and validation to ensure accuracy and quality Collaborate … with stakeholders to gather requirements and deliver data solutions Conduct A/B tests and statistical analysis to inform business decisions Monitor key performance indicators (KPIs) and provide regular performance reports Work with data engineering teams to improve data pipelines and data architecture Present findings to stakeholders through visualisations and presentations Training … All work uploaded to Aptem Monthly college release days (online or face to face) Potential full-time position Apprenticeship Standard Data analyst (level 4) Training Provider BRADFORD COLLEGE Working Week Monday - Thursday 07:30am - 16:15. Friday 07:30am - 12.45pm. Expected Duration 2 Years Positions Available 1 Closing Date Thursday, 14th August 2025 Start Date Wednesday, 20th August More ❯

Senior Data Engineers with a background are required by this major client, who is creating a new start up engineering capability in their Leeds offices, where you will provide best in class Data Engineering services to a wide range of major Public Sector organisations. As a result of the work that they do, this client requires … national or dual UK national. Please note your application will not be taken forward if you cannot fulfil these requirements. In order to secure one of these Senior Data Engineer roles you must be able to demonstrate the following experience: Experience in prominent languages such as Python, Scala, Spark, SQL. Good experience in using Databricks Experience working with … Knowledge and experience using at least one Data Platform Technology such as Quantexa, or Palantir Knowledge of test automation frameworks and ability to automate testing within the pipeline This is an opportunity for significant growth, but with a start-up feel and mentality. In this role you will have the chance to build and be part of More ❯

Comfortable providing technical guidance and helping resolve ambiguity on large-scale work. Preferred Qualifications Experience working with Gemini models and GCP-native AI tooling. Familiarity with real-time data pipelines or streaming architectures. Prior work on context-aware or intelligent product experiences. Experience with design systems or component libraries in a production setting. Passion for continuously learning and More ❯