1 to 25 of 75 Permanent Data Pipeline Jobs in South Yorkshire

Junior Data Engineer – Manufacturing - £30-35k – Hybrid or Remote Overview: An exciting opportunity has arisen for a Junior Data Engineer to join one of the world’s leading transportation manufacturers who specialise in providing safety and securement solutions for mobility devices. They are known best … the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business … -related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency More ❯

the ease of use - they have introduced several industry-first products and maintain a strong focus on research and development. As a Junior Data Engineer, you will play a crucial role in developing the data infrastructure that supports the analytics and operational functions of our business … enterprise. Role & Responsibilities: Data Infrastructure & Support: Provide first-line support for data engineering tasks, such as managing and monitoring data pipelines, resolving issues, and ensuring data integrity. Work with both structured and unstructured datasets to design and implement data models … -related errors. Collaborate with business stakeholders to identify data requirements and deliver sustainable solutions. Monitor and troubleshoot data pipeline issues to maintain data integrity and accuracy. Assist in the development, maintenance, and optimization of ETL (Extract, Transform, Load) processes for efficiency More ❯

make better use of it. We provide advice, support and delivery services across various industry sectors covering a multitude of areas, ranging from Data Strategy, Governance and Security to Data Platform Modernisation, Cloud and Customer Intelligence. We do everything with a view to creating tangible business … measure how we are performing and ensures we deliver the value that our customers deserve. Summary We are seeking a skilled and proactive Data Engineer with experience in Microsoft Fabric to join our growing data team. You will be responsible for developing, maintaining, and optimising data … lakehouse architectures. Hands-on experience with Power BI for data integration and visualisation. Familiarity with DevOps practices, particularly around data pipeline deployment. Good problem-solving skills and the ability to work under minimal supervision. Effective communicator with stakeholders of varying technical expertise. Benefits Annual Leave More ❯

make better use of it. We provide advice, support and delivery services across various industry sectors covering a multitude of areas, ranging from Data Strategy, Governance and Security to Data Platform Modernisation, Cloud and Customer Intelligence. We do everything with a view to creating tangible business … measure how we are performing and ensures we deliver the value that our customers deserve. Summary We are seeking a skilled and proactive Data Engineer with experience in Microsoft Fabric to join our growing data team. You will be responsible for developing, maintaining, and optimising data … lakehouse architectures. Hands-on experience with Power BI for data integration and visualisation. Familiarity with DevOps practices, particularly around data pipeline deployment. Good problem-solving skills and the ability to work under minimal supervision. Effective communicator with stakeholders of varying technical expertise. Benefits Annual Leave More ❯

Azure Data Engineer/Consultant – Up to £75K + Bonus Location: Remote (UK-based, with very occasional travel) Salary: Up to £75,000 + bonus Company & role Join one of the UK’s most advanced Microsoft partners, specialising in Azure-led Data & AI solutions. This consultancy … is known for transforming data into tangible value through innovative platforms and strategic data services. With a collaborative team of experts, they deliver impactful solutions across enterprise clients, leveraging the very latest in Microsoft technology. Why This Role Stands Out Play a key role in shaping … across the Azure Data Platform (Fabric, Synapse, Databricks, Data Lake, Data Factory, Purview, etc.) Strong data pipeline and ETL skills, including data modelling and transformation Experience implementing Medallion Architecture (Bronze, Silver, Gold layers) in lakehouse or Databricks environments Familiarity More ❯

Azure Data Engineer/Consultant – Up to £75K + Bonus Location: Remote (UK-based, with very occasional travel) Salary: Up to £75,000 + bonus Company & role Join one of the UK’s most advanced Microsoft partners, specialising in Azure-led Data & AI solutions. This consultancy … is known for transforming data into tangible value through innovative platforms and strategic data services. With a collaborative team of experts, they deliver impactful solutions across enterprise clients, leveraging the very latest in Microsoft technology. Why This Role Stands Out Play a key role in shaping … across the Azure Data Platform (Fabric, Synapse, Databricks, Data Lake, Data Factory, Purview, etc.) Strong data pipeline and ETL skills, including data modelling and transformation Experience implementing Medallion Architecture (Bronze, Silver, Gold layers) in lakehouse or Databricks environments Familiarity More ❯

collaboration, recognition and inclusivity and offer a wide range of benefits to support staff wellbeing. Your Future Starts Here PURPOSE OF JOB: The Data & AI Solutions Deployment Architect will be instrumental in architecting and deploying Azure Data & AI Solutions. A solid, technical, ‘hands-on’ (not just … grasp of Azure Fabric is essential as we navigate clients through all of the infrastructure, coding, and product development lifecycle challenges around successful Data and AI implementation and deployment. This customer-centric position involves delivering projects, conducting workshops and physically working on client’s environments via Python, and … C#. This role would suit an experienced Data platform engineer or Data Solutions Architect who thrives in working across multiple projects with a diverse range of clients, and who places the importance of bridging the gap between complex data problems and real life product More ❯

collaboration, recognition and inclusivity and offer a wide range of benefits to support staff wellbeing. Your Future Starts Here PURPOSE OF JOB: The Data & AI Solutions Deployment Architect will be instrumental in architecting and deploying Azure Data & AI Solutions. A solid, technical, ‘hands-on’ (not just … grasp of Azure Fabric is essential as we navigate clients through all of the infrastructure, coding, and product development lifecycle challenges around successful Data and AI implementation and deployment. This customer-centric position involves delivering projects, conducting workshops and physically working on client’s environments via Python, and … C#. This role would suit an experienced Data platform engineer or Data Solutions Architect who thrives in working across multiple projects with a diverse range of clients, and who places the importance of bridging the gap between complex data problems and real life product More ❯

Senior Data Engineer Bristol 6-month contract Paying up to £80p/h (Outside IR35) *** Please note that due to the nature of the work you must hold or be eligible to obtain DV level security clearance *** Responsibilities : Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. Implement data ingestion, transformation, and integration processes, ensuring data quality and security. Collaborate with data architects and security teams to ensure compliance with security policies and data governance standards. … Manage and monitor large-scale data flows in real-time, ensuring system performance, reliability, and data integrity. Develop robust data models to support analytics and reporting within secure environments. Perform troubleshooting, debugging, and performance tuning of data pipelines and the Elastic Stack. More ❯

Senior Data Engineer Bristol 6-month contract Paying up to £80p/h (Outside IR35) *** Please note that due to the nature of the work you must hold or be eligible to obtain DV level security clearance *** Responsibilities : Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. Implement data ingestion, transformation, and integration processes, ensuring data quality and security. Collaborate with data architects and security teams to ensure compliance with security policies and data governance standards. … Manage and monitor large-scale data flows in real-time, ensuring system performance, reliability, and data integrity. Develop robust data models to support analytics and reporting within secure environments. Perform troubleshooting, debugging, and performance tuning of data pipelines and the Elastic Stack. More ❯

our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the More ❯

Data Engineer We’re working with a forward-thinking Microsoft partnered consultancy who are looking for a Data Engineer to join their growing team. You will have the opportunity to work with some of the latest Microsoft technologies with a focus on projects on Databricks implementations. … You will join a team at the centre of a number of data-driven projects where you will be responsible for the design, development and creation of data solutions. You will work on the full end-to-end product lifecycle from platform design to insights creation. … This is a fantastic opportunity for someone who is passionate about building scalable, high-performance data solutions utilising the latest cutting-edge cloud technologies. As part of this role, you will be responsible for some of the following areas • Design and develop scalable data pipelines to More ❯

Data Engineer We’re working with a forward-thinking Microsoft partnered consultancy who are looking for a Data Engineer to join their growing team. You will have the opportunity to work with some of the latest Microsoft technologies with a focus on projects on Databricks implementations. … You will join a team at the centre of a number of data-driven projects where you will be responsible for the design, development and creation of data solutions. You will work on the full end-to-end product lifecycle from platform design to insights creation. … This is a fantastic opportunity for someone who is passionate about building scalable, high-performance data solutions utilising the latest cutting-edge cloud technologies. As part of this role, you will be responsible for some of the following areas • Design and develop scalable data pipelines to More ❯

a wide range of benefits to support staff wellbeing. Your Future Starts Here PURPOSE OF JOB: We’re looking for an experienced Azure Data & AI Engineer with a strong focus on advanced analytics, machine learning, and applied AI - particularly Generative AI. This role will suit a technically capable … Insights or Log Analytics. KEY RESPONSIBILITIES: Have 5+ years of technical consulting or a similar chronology of demonstrable success in coding and deploying Data and AI models onto Azure environments. At least one year using Azure Fabric Create scalable and reliable data pipelines for data processing, transformation, and storage. Establish and enforce data governance policies and be ‘hands-on’ in exploratory data analysis to identify and measure quality gaps that are a block to Data and Ai implementation. Implement robust security measures to protect sensitive data More ❯

a wide range of benefits to support staff wellbeing. Your Future Starts Here PURPOSE OF JOB: We’re looking for an experienced Azure Data & AI Engineer with a strong focus on advanced analytics, machine learning, and applied AI - particularly Generative AI. This role will suit a technically capable … Insights or Log Analytics. KEY RESPONSIBILITIES: Have 5+ years of technical consulting or a similar chronology of demonstrable success in coding and deploying Data and AI models onto Azure environments. At least one year using Azure Fabric Create scalable and reliable data pipelines for data processing, transformation, and storage. Establish and enforce data governance policies and be ‘hands-on’ in exploratory data analysis to identify and measure quality gaps that are a block to Data and Ai implementation. Implement robust security measures to protect sensitive data More ❯

Integration · Design, build and maintain AI-driven information retrieval and generation workflows · Securely integrate and manage external and internal AI services · Develop advanced data matching and search solutions · Architect effective interaction frameworks and context-management strategies · Collaborate on extending core model capabilities and protocols Data Engineering … Design and maintain data pipelines for efficient data processing · Ensure efficient data flow between systems and LLM components · Implement data validation, transformation, and enrichment processes · Optimise data storage and retrieval for AI/ML workloads Security & Compliance · Implement security best … understanding of software design principles and coding standards Preferred Skills and Capabilities · Experience with vector databases and embedding models · Knowledge of data pipeline tools and ETL processes · Familiarity with React or other frontend technologies for better cross-team collaboration · Experience implementing LLM integration frameworks · Background in pharmaceutical More ❯

Integration · Design, build and maintain AI-driven information retrieval and generation workflows · Securely integrate and manage external and internal AI services · Develop advanced data matching and search solutions · Architect effective interaction frameworks and context-management strategies · Collaborate on extending core model capabilities and protocols Data Engineering … Design and maintain data pipelines for efficient data processing · Ensure efficient data flow between systems and LLM components · Implement data validation, transformation, and enrichment processes · Optimise data storage and retrieval for AI/ML workloads Security & Compliance · Implement security best … understanding of software design principles and coding standards Preferred Skills and Capabilities · Experience with vector databases and embedding models · Knowledge of data pipeline tools and ETL processes · Familiarity with React or other frontend technologies for better cross-team collaboration · Experience implementing LLM integration frameworks · Background in pharmaceutical More ❯

Data Quality and Governance Manager - Hybrid - Leeds - £65,000 My client is looking for a Data Quality & Governance Manager to shape how data is trusted and used throughout their organisation. They are looking for someone to lead efforts in ensuring the consistency, reliability, and … governance of data. This role will work closely with business and technical teams to develop and implement frameworks for data quality, governance, and master data management, using Snowflake and Microsoft Azure. Requirements: Experience in data governance and data quality management Strong knowledge … of data warehouse architecture and cloud platforms, especially Snowflake and Microsoft Azure. Proven experience designing and managing data pipelines in cloud environments with integrated data governance principles. Hands-on experience with data migrations Strong experience in working with data engineers More ❯

Data Engineer Remote | Contract | Outside IR35 We are a fast-growing technology company building a cutting-edge business identity platform. Our solution enables banks, fintechs, and enterprises to seamlessly verify, onboard, and monitor businesses of all sizes. We’re looking for a skilled Data Engineer to … help scale and enhance our clients platform and data infrastructure. In this role, you’ll be instrumental in building robust data pipelines, ensuring data quality, and supporting a reliable, cloud-native data platform. What You’ll Do: Design, build, and maintain scalable … data pipelines and ETL/ELT workflows Work with Snowflake , SQL , and Python to transform and manage large datasets Leverage AWS services (e.g., S3, Lambda, Glue) to build modern, cloud-based data solutions Collaborate with cross-functional teams to improve data accessibility and quality More ❯

Data Engineer Remote | Contract | Outside IR35 We are a fast-growing technology company building a cutting-edge business identity platform. Our solution enables banks, fintechs, and enterprises to seamlessly verify, onboard, and monitor businesses of all sizes. We’re looking for a skilled Data Engineer to … help scale and enhance our clients platform and data infrastructure. In this role, you’ll be instrumental in building robust data pipelines, ensuring data quality, and supporting a reliable, cloud-native data platform. What You’ll Do: Design, build, and maintain scalable … data pipelines and ETL/ELT workflows Work with Snowflake , SQL , and Python to transform and manage large datasets Leverage AWS services (e.g., S3, Lambda, Glue) to build modern, cloud-based data solutions Collaborate with cross-functional teams to improve data accessibility and quality More ❯

and LATAM. We are about people who create bold things, who make a difference, who have fun, and who love their work. Big Data & Analytics Center of Excellence, data consulting and data engineering branch at SoftServe. Starting as a group of three enthusiasts back … in 2013, hundreds of Data Engineers and Architects nowadays build Data & Analytics end-to-end solutions from strategy through technical design and PoC to full-scale implementation. We have customers in Healthcare, Finance, Manufacturing, Retail, and Energy domains. We hold top-level partnership statuses with all … and others. IF YOU ARE Experienced with Python/PySpark Proficient working with Databricks Lakehouse architecture and principles Having 2+ years of designing data models, building ETL pipelines, and wrangling data to solve business problems Experienced with Azure cloud technologies Modern Data Estate such More ❯

and LATAM. We are about people who create bold things, who make a difference, who have fun, and who love their work. Big Data & Analytics Center of Excellence, data consulting and data engineering branch at SoftServe. Starting as a group of three enthusiasts back … in 2013, hundreds of Data Engineers and Architects nowadays build Data & Analytics end-to-end solutions from strategy through technical design and PoC to full-scale implementation. We have customers in Healthcare, Finance, Manufacturing, Retail, and Energy domains. We hold top-level partnership statuses with all … and others. IF YOU ARE Experienced with Python/PySpark Proficient working with Databricks Lakehouse architecture and principles Having 2+ years of designing data models, building ETL pipelines, and wrangling data to solve business problems Experienced with Azure cloud technologies Modern Data Estate such More ❯

Kingdom) About The Company: We have partnered with a company that empowers underwriters to serve their insureds more effectively. They are using advanced data intelligence tools to rebuild the way that underwriters share and exchange risk. With a current focus on the small and medium-sized businesses that … into insurance pools, and their speciality portfolio is fully diversified with very low catastrophe, aggregation or systemic risk. The Role: Designing and implementing data pipelines and models, ensuring data quality and integrity. Solving challenging data integration problems, utilising optimal patterns, frameworks, query techniques, sourcing … from vast and varying data sources. Building, maintaining, and optimising our Data Warehouse to support reporting and analytics needs. Collaborating with product managers, business stakeholders and engineers to understand the data needs, representing key data insights in a meaningful way. Staying up More ❯

Kingdom) About The Company: We have partnered with a company that empowers underwriters to serve their insureds more effectively. They are using advanced data intelligence tools to rebuild the way that underwriters share and exchange risk. With a current focus on the small and medium-sized businesses that … into insurance pools, and their speciality portfolio is fully diversified with very low catastrophe, aggregation or systemic risk. The Role: Designing and implementing data pipelines and models, ensuring data quality and integrity. Solving challenging data integration problems, utilising optimal patterns, frameworks, query techniques, sourcing … from vast and varying data sources. Building, maintaining, and optimising our Data Warehouse to support reporting and analytics needs. Collaborating with product managers, business stakeholders and engineers to understand the data needs, representing key data insights in a meaningful way. Staying up More ❯

Lead Data Engineer Telford, Shropshire Up to £100k + Benefits A new role for a Lead Data Engineer has arisen with an international Tech company working on a major UK Government Programme designing and delivering a range of data solutions. This is an opportunity … to play a key role in a collaborative function architecting data pipelines and infrastructure, leading in both a technical sense but also through mentorship and establishing best practices and standards. The Role The Lead Data Engineer will design and implement complex data flows to … connect operational systems, build and maintain data environments, evaluate and develop tools to improve data analytics platforms and recognise opportunities to reuse existing data flows between teams. Requirements This is a leadership role and requires not only demonstrable experience of leading on data More ❯