12 of 12 Data Pipeline Jobs in Hertfordshire

Social network you want to login/join with: As a Data Scientist, you'll join our dynamic Business Performance & Insights team, working alongside talented data analysts, engineers, and subject matter experts. You'll deliver high-impact machine learning models and analytics solutions that directly influence business strategy and service performance. This is more than a … and communities, and to make better use of water and safeguard the local environment now and in the future. We're on a journey to become a truly data-driven organisation. With data at the heart of how we serve our customers and drive performance, we are building a culture of innovation and insight-led decision … making across the business. Key Responsibilities Design and develop advanced analytics and machine learning models to solve real-world business challenges. Work across departments to identify opportunities where data science can deliver measurable value. Create powerful visualisations and present findings to technical and non-technical stakeholders. Build, deploy, and maintain robust data pipelines and production-grade More ❯

Data Team Lead Role Overview: We are currently looking for a Data Team Lead to head our data operations and strategy.This is a hands-on leadership role responsible for managing a small but growing data team, ensuring the delivery of high-quality data infrastructure, reporting, and business intelligence tools to … support cross-functional goals.The ideal candidate will have a strong technical background, experience in team management, and the ability to align data processes with business objectives. Key Responsibilities: Lead and develop the internal data team, including hiring, mentoring, and performance management Build and maintain scalable data pipelines, storage solutions, and reporting dashboards Ensure data accuracy, quality, governance, and compliance with relevant standards Translate business needs into technical requirements and data-driven solutions Collaborate with marketing, product, and executive teams to support decision-making Monitor performance metrics and proactively identify areas for optimisation Own and refine the company's data strategy, aligned with broader business goals Requirements: Minimum More ❯

Stevenage As a data engineer specialising in generative AI ; this role will see you working in a developing international and transversal structure. You will have the responsibility to evaluate, build and maintain data sets for internal customers whilst ensuring they can be maintained. Salary : Circa £45,000 - £55,000 depending on experience Dynamic (hybrid) working … treatments Facilities : Fantastic site facilities including subsidised meals, free car parking and much more... The opportunity: The MBDA IM GenAI delivery Office department is looking for an experienced data engineer able to evaluate design, deploy, improve and support MBDA data sets. You will ensure MBDA data pipelines are designed to be resilient, secure and … responsive. You will use your data engineering expertise to collaborate with different internal customers regarding their data, ensuring they are optimised and secured for their needs. You will provide your knowledge in data management and data quality to guarantee compliance to MBDA data governance. A key part of this role More ❯

Role Overview: We are currently looking for a Data Team Lead to head our data operations and strategy.This is a hands-on leadership role responsible for managing a small but growing data team, ensuring the delivery of high-quality data infrastructure, reporting, and business intelligence tools to support cross-functional goals.The ideal … candidate will have a strong technical background, experience in team management, and the ability to align data processes with business objectives. Key Responsibilities: Lead and develop the internal data team, including hiring, mentoring, and performance management Build and maintain scalable data pipelines, storage solutions, and reporting dashboards Ensure data accuracy, quality, governance … and compliance with relevant standards Translate business needs into technical requirements and data-driven solutions Collaborate with marketing, product, and executive teams to support decision-making Monitor performance metrics and proactively identify areas for optimisation Own and refine the company's data strategy, aligned with broader business goals Requirements: Minimum 5 years of experience in data More ❯

This is a pivotal role in a growing energy business with serious ambitions around data. You’ll shape how data is collected, governed, and used to drive decisions — across operations, risk, and customer delivery. It’s hands-on, strategic, and full of opportunity to lead from the front. What You’ll Be Doing: Designing and developing robust data pipelines to centralise and clean data from multiple sources Building conceptual and physical data models to support reporting and analysis Championing data governance and implementing controls to ensure quality and compliance Developing automated ETL solutions using Microsoft Azure tools Leading best practice in engineering standards, code quality, and peer review Supporting teams … through training sessions and knowledge sharing Main Skills Needed: Proven experience in data engineering, ideally 5+ years Advanced SQL skills and strong knowledge of Azure (Data Lake, Synapse, SQL Server) Comfortable with Python, C#, and working with APIs Familiar with cloud data platforms (Azure essential; AWS useful) Confident designing data architecture and More ❯

Data Engineer (Generative AI) - Defence & Technology - British Citizens only Are you a Data Engineer ready to build and secure the next generation of data pipelines for cutting-edge Generative AI projects? We're working with a highly respected, multinational defence and technology company looking for a skilled Data Engineer to join a … a unique opportunity to directly shape the technology roadmap and deliver critical, high-impact solutions. In this role, you will be responsible for evaluating, designing, and maintaining resilient data sets, collaborating with internal customers to optimise their data for security and performance. A key part of your work will involve staying ahead of new technology trends … the company remains at the forefront of Generative AI. What we're looking for: Expertise in SQL and noSQL technologies (e.g., MS SQL, Oracle, MongoDB). Experience with data processing and exchange (ETL, ESB, API). Strong development skills, particularly with Python. Big data experience/knowledge (hadoop etc) Advantageous to have knowledge of Generative AI More ❯

A fantastic opportunity has arisen for a person to join a department to assist with the delivery of innovative generative AI data solutions. ** Please note that due to the nature of the client’s business, only candidates who currently hold full sole British Citizenship (without limitations) will be considered. ** This role performs the duties of a Data Engineer specialising in generative AI and reports to the senior lead within a developing international and transversal structure. Key Responsibilities: Evaluate, design, deploy, improve and support data sets for internal customers, ensuring resilience, security and responsiveness. Collaborate with internal stakeholders to optimise and secure their data in line with operational needs. Maintain and enhance … data pipelines, ensuring compliance with data governance standards. Provide expertise in data management and quality assurance. Stay up to date with evolving technology, contribute to the technology roadmap, and deliver cutting-edge solutions. The successful candidate will ideally have: Knowledge in NLP (Natural Language Processing). Knowledge in OCR (Object Character Recognition). Knowledge More ❯

develop simulation tools to support product integration and create software that enhances manufacturing processes by streamlining and supporting operations. Key Responsibilities: Design and develop real-time test and data capture software on Linux using C++, with a focus on performance and reliability. Build and maintain scalable, multi-threaded applications that interface with high-speed SFPDP cards across multi … channel data pipelines. Define and implement deterministic data sequencing and synchronization logic to support accurate capture and replay functionality. Collaborate with hardware teams to ensure seamless integration and precise timing alignment between software and capture systems. Lead and contribute to system-level integration, verification, and validation, ensuring robust performance across all components. Perform in-depth performance … specifications, and test procedures to support knowledge transfer and formal testing activities. What do you need?: You'll support the development and integration of real-time test and data capture systems in a Linux environment, using C++ (C# is a plus), high-speed SFPDP cards, and multi-threaded pipelinescontributing to software design, verification, and system validation, alongside the More ❯

develop simulation tools to support product integration and create software that enhances manufacturing processes by streamlining and supporting operations. Key Responsibilities: Design and develop real-time test and data capture software on Linux using C++, with a focus on performance and reliability. Build and maintain scalable, multi-threaded applications that interface with high-speed SFPDP cards across multi … channel data pipelines. Define and implement deterministic data sequencing and synchronization logic to support accurate capture and replay functionality. Collaborate with hardware teams to ensure seamless integration and precise timing alignment between software and capture systems. Lead and contribute to system-level integration, verification, and validation, ensuring robust performance across all components. Perform in-depth performance … specifications, and test procedures to support knowledge transfer and formal testing activities. What do you need?: You'll support the development and integration of real-time test and data capture systems in a Linux environment, using C++ (C# is a plus), high-speed SFPDP cards, and multi-threaded pipelinescontributing to software design, verification, and system validation, alongside the More ❯

develop simulation tools to support product integration and create software that enhances manufacturing processes by streamlining and supporting operations. Key Responsibilities: Design and develop real-time test and data capture software on Linux using C++, with a focus on performance and reliability. Build and maintain scalable, multi-threaded applications that interface with high-speed SFPDP cards across multi … channel data pipelines. Define and implement deterministic data sequencing and synchronization logic to support accurate capture and replay functionality. Collaborate with hardware teams to ensure seamless integration and precise timing alignment between software and capture systems. Lead and contribute to system-level integration, verification, and validation, ensuring robust performance across all components. Perform in-depth performance … specifications, and test procedures to support knowledge transfer and formal testing activities. What do you need?: You'll support the development and integration of real-time test and data capture systems in a Linux environment, using C++ (C# is a plus), high-speed SFPDP cards, and multi-threaded pipelinescontributing to software design, verification, and system validation, alongside the More ❯

British Citizenship Overview A leading engineering organisation is seeking an experienced C++ Software Engineer to join a development team responsible for building and integrating real-time test and data capture systems. The work is embedded in a technically demanding environment, involving high-speed digital interfaces, deterministic data flows, and complex multi-threaded processing pipelines. This is … The successful candidate will play a key role in enabling advanced test and validation capabilities for internal engineering teams. Key Responsibilities Design and develop real-time software for data capture and test systems using C++ in a Linux environment Interface with high-speed SFPDP cards and manage multi-channel data pipelines Build scalable and maintainable multi … threaded applications Implement deterministic data sequencing and synchronisation logic Collaborate with hardware teams for seamless integration and timing alignment Lead and support system-level integration, verification, and testing Conduct performance optimisation, profiling, and debugging Document architecture, interface specifications, and test procedures Required Skills & Experience Strong experience in C++ development on Linux platforms Proficiency in C# for tooling or More ❯

be involved in: Design and develop real-time test and capture software on Linux systems using C++ Integrate and interface with high-speed SFPDP cards across multi-channel data pipelines Architect and implement robust, scalable, and maintainable multi-threaded applications Define and implement data sequencing and synchronisation logic to support deterministic capture and replay Work with … Your skillset may include: Experience in C++ development in a Linux environment Proficiency in C# for tooling, test interfaces, or Windows components Proven experience working with real-time data systems, preferably involving SFPDP or other high-speed digital protocols Demonstrable skills in multi-threaded and concurrent programming Experience with low-level I/O, DMA, or driver-level … Familiarity with software sequencing frameworks or custom state machines for test execution Experience with version control systems (e.g., Git) and CI/CD pipelines Knowledge of FPGA-based data acquisition systems and synchronisation methods Familiarity with unit testing frameworks, automated test rigs, or hardware-in-the-loop (HWIL) setups Experience developing or integrating with real-time operating systems More ❯