21 of 21 Data Pipeline Jobs in the Thames Valley

Engineer, you will operate within an Agile delivery environment, working closely with the Data Product Manager and Data Architect to ensure your team maintains a pipeline of delivery against the Backlog; providing vital insight from our wide-ranging dataset to support executive and operational decision making that will underpin sustained growth of BFS business units … solutions. You will also have management responsibilities for a small team of Data Engineers, who you will coach, support and organise to ensure we sustain a predictable pipeline of delivery, whilst ensuring all appropriate governance and best practice is adhered to. Your responsibilities as our Senior Databricks Engineer will include: Understand the business/product strategy and … Azure DevOps, collaborating on Sprint cycles, code deployment, version control, and development practices Develop and maintain data models, schemas, and documentation Monitor and troubleshoot data pipeline issues, ensuring timely resolution Stay up-to-date with the latest industry trends and technologies in data engineering and recommend improvements to existing systems or processes as More ❯

Engineer, you will operate within an Agile delivery environment, working closely with the Data Product Manager and Data Architect to ensure your team maintains a pipeline of delivery against the Backlog; providing vital insight from our wide-ranging dataset to support executive and operational decision making that will underpin sustained growth of BFS business units … solutions. You will also have management responsibilities for a small team of Data Engineers, who you will coach, support and organise to ensure we sustain a predictable pipeline of delivery, whilst ensuring all appropriate governance and best practice is adhered to. Your responsibilities as our Senior Databricks Engineer will include: Understand the business/product strategy and … Azure DevOps, collaborating on Sprint cycles, code deployment, version control, and development practices Develop and maintain data models, schemas, and documentation Monitor and troubleshoot data pipeline issues, ensuring timely resolution Stay up-to-date with the latest industry trends and technologies in data engineering and recommend improvements to existing systems or processes as More ❯

hunt for builders . No, we've not ventured into construction in our quest to conquer the world, rather a designer and builder of systems for all things data related where we are conquering the Data World. We can offer an interesting insight into projects spanning a variety of sectors, which may include industries such as … to explore our dynamic team where you can truly act as an expert in your field in support of our clients and their challenges in the world of data and technology, read on! The Team We're Intuita - a fast growing consultancy that's making waves in both the consultancy and technology space. With our ambitious goals for … in the data pipelines. • Stay updated with the latest Azure technologies and industry best practices to continuously enhance data engineering capabilities. Looking at our current pipeline of work, we can also consider those with an Analytics Engineering lean, with experience within BigQuery (GCP) with data modelling in DBT and mobile/telecoms industry More ❯

Role: Head of Data Management Location: Milton Keynes (Hybrid 2 Days On-site Weekly) Salary: £55,000 - £70,000 Benefits: Bonus, Company Cars, Flexible Benefit Package Network IT is recruiting for a Head of Data Management to join a leading automotive brand's UK IT function. In this senior role, youll lead a high-performing data management team, drive technical delivery across enterprise data platforms, and ensure compliance, scalability, and performance within a global standards-based ecosystem. Reporting to the Head of IT, youll take ownership of operational excellence, stakeholder engagement, and vendor performance across a broad data estate including Azure Data Factory and Data Lake … Strong experience in IT service management, strategy planning, and business process alignment. Hands-on knowledge of Azure Data Factory, Data Lake , and data pipeline architecture. Proven experience managing data security, access, and regulatory compliance. Familiarity with automation tools (AI, RPA) and Agile project delivery (e.g. Scrum, Kanban). Demonstrated ability to More ❯

Role: Data Team Lead Location: Milton Keynes (Hybrid 3 Days On-site Weekly) Salary: £55,000 - £65,000 Benefits: Bonus, Company Cars, Flexible Benefit Package Network IT is recruiting for a Data Team Lead to join a leading organisations UK IT function. This role will see you managing and developing a high-performing data team, while ensuring the delivery of effective and scalable data solutions. Youll play a key role in setting direction, building capability, and aligning data initiatives with business priorities. The role requires strong leadership, stakeholder management, and the ability to oversee data programmes from strategy through to delivery. Key Responsibilities: Lead, mentor, and … data leadership roles within complex organisations. Proven track record of leading and developing technical teams. Good understanding of data warehousing, ETL, and data pipeline solutions. Experience with data governance, compliance, and security frameworks. Strong stakeholder engagement, vendor management, and planning skills. Experience managing budgets and resources effectively. Excellent communication, leadership, and More ❯

Join us as a Data Platform Engineer for our client. At Peregrine, we re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector we are always in search of the best … you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries. The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and onpremises … with over 3 years of relevant experience in data engineering, platform engineering, or a related field, with demonstrated hands-on expertise in NiFi and data pipeline design in regulated environments. Responsibilities: Design, develop, and maintain robust and secure data pipelines using NiFi and related big data technologies. • Troubleshoot and optimize NiFi More ❯

Join us as a Data Platform Engineer for our client. At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector – we are always in search of the best … you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries. The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and onpremises … with over 3 years of relevant experience in data engineering, platform engineering, or a related field, with demonstrated hands-on expertise in NiFi and data pipeline design in regulated environments. Responsibilities: Design, develop, and maintain robust and secure data pipelines using NiFi and related big data technologies. • Troubleshoot and optimize NiFi More ❯

to-end testing of data ingestion, parsing, aggregation, and schema validation processes in AWS. Conduct stress testing to evaluate the performance of the entire data pipeline built on AWS. Develop and execute test plans, test cases, and test scripts to ensure data quality and integrity. Collaborate with data engineers and developers … to identify and resolve data quality issues. Monitor data quality metrics and implement improvements to enhance data reliability. Validate data transformations and ensure consistency across various data sources. Document test results and provide detailed reports on data quality findings. Required Skills and Experience: Proven experience with AWS services … such as EMR, Lambda, Redshift, Firehose, S3, Iceberg, Athena, and DynamoDB. Strong understanding of data ingestion, parsing, aggregation, and schema validation processes. Proficiency in SQL for data querying and validation. Experience with Python for scripting and automation. Familiarity with data quality tools and frameworks. Excellent analytical and problem-solving skills. Strong attention to detail More ❯

every decision wemake and action we take, guiding us to deliver impact how and where it mattersmost . Connect to your opportunity We're looking for a skilled Data Lake Reporting Assistant Manager to join our team and play a key role in developing and maintaining our data lake reporting environment. You'll work closely with … experienced data analysts and engineers to deliver impactful reports that help us understand our business better. Key responsibilities include: Collaborate closely with senior data analysts and engineers, contributing to the development and implementation of our reporting strategy. Learn and apply industry best practices for data warehousing, reporting, and visualization. Data Lake Reporting … Environment: Become proficient in navigating and understanding data lake architecture. Assist in implementing data quality checks and validation processes to ensure accurate and reliable reporting. Work alongside data engineers to understand and optimize data pipelines for reporting needs. Translate business requirements into technical specifications for reports and dashboards, working with senior team More ❯

Data Consultant - ETL Projects UK - Remote | Permanent Up to £55,000 + Benefits Are you a Data Consultant with a strong background in hands-on ETL development and data transformation? Are you looking to lead technically complex data migration projects that genuinely impact digital services across Local Government? We're hiring for … an experienced Data Consultant to join a growing team delivering mission-critical data migration projects. You'll be part of an expert unit supporting onboarding for a new cloud platform - working directly with raw datasets from legacy systems and transforming them into future-ready digital infrastructure. What You'll Be Doing: Leading data migration … and ETL projects for UK public sector software implementations Analysing and mapping legacy database schemas (Oracle, SQL Server, Postgres) Extracting raw data from legacy systems, often with limited documentation Writing and executing SQL scripts and/or using ETL tools to build data pipelines Creating transformation logic and performing reconciliation and validation Working closely with customers More ❯

Integration Engineer Mainly Remote (monthly meet ups with team) Outside IR35 Experienced Integration Engineer needed to help design and develop data integration solutions using Informatica IPaaS and Snowflake data Lakehouse You will Design, develop and implement ELT & ETL pipelines and workflows using Informatica PowerCenter and Informatica Cloud. Build connectors from multiple sources including Cloud platforms, bespoke … and out of the box APIs, databases, and flat files. Develop and maintain data mappings, transformations, data pipelines and integration workflows. Implement real-time and … batch data integration solutions to support business requirements Ensure any integration pipelines or ELT/ETL workflows are built with error and exception handling built into the pipeline/workflow. Your experience Proven experience in a similar role so you can provide advice and guidance to the team Excellent experience in Informatica ELT & ETL development and integration More ❯

Integration Engineer Mainly Remote (monthly meet ups with team) Outside IR35 Experienced Integration Engineer needed to help design and develop data integration solutions using Informatica IPaaS and Snowflake data Lakehouse You will Design, develop and implement ELT & ETL pipelines and workflows using Informatica PowerCenter and Informatica Cloud. Build connectors from multiple sources including Cloud platforms, bespoke … and out of the box APIs, databases, and flat files. Develop and maintain data mappings, transformations, data pipelines and integration workflows. Implement real-time and … batch data integration solutions to support business requirements Ensure any integration pipelines or ELT/ETL workflows are built with error and exception handling built into the pipeline/workflow. Your experience Proven experience in a similar role so you can provide advice and guidance to the team Excellent experience in Informatica ELT & ETL development and integration More ❯

JoinusasaDataPlatformEngineerforourclient. AtPeregrine,werealwaysseekingSpecialistTalentthathavetheidealmixofskills,experience,and attitude,toplacewithourvastarrayofclients.FromBusinessAnalystsinlargegovernment organisationstoSoftwareDevelopersintheprivatesectorwearealwaysinsearchofthebest talenttoplace,now. HowSpecialistTalentWorks AtPeregrine,wefindthebesttalentforourclients.AsapermanentemployeeofPeregrine,with accesstoallourstandardbenefits,youwillbedeployedacrossourportfolioofclientsasaspecialistconsultant,workingonawidearrayofcomplexprojectsacrossmultipleindustries. TheRole: AsaDataPlatformEngineerinahighlyregulatedenvironment,youwillberesponsiblefordesigning,building,andmaintainingsecureandscalabledatainfrastructurethatsupportsbothcloudandonpremisesplatforms.Youwillplayakeyroleinensuringthatalldatasystemscomplywithindustryregulationsandsecuritystandardswhileenablingefficientaccessforanalyticsandoperationalteams. AstrongcommandofApacheNiFiisessentialforthisrole.Youwillbeexpectedtodesign, implement,andmaintaindataflowsusingNiFi,ensuringaccurate,efficient,andsecuredata ingestion,transformation,anddelivery.Youshouldbeadeptatidentifyingandresolvingissues withinNiFiflows,managingperformancebottlenecks,andimplementingrobusterrorhandling strategies. Youllworkcloselywithcross-functionalteamsincludingdataarchitects,complianceofficers,and cybersecurityspecialiststointegratedatafromvarioussystemssuchasdatabases More ❯

JoinusasaDataPlatformEngineerforourclient. AtPeregrine,werealwaysseekingSpecialistTalentthathavetheidealmixofskills,experience,and attitude,toplacewithourvastarrayofclients.FromBusinessAnalystsinlargegovernment organisationstoSoftwareDevelopersintheprivatesectorwearealwaysinsearchofthebest talenttoplace,now. HowSpecialistTalentWorks AtPeregrine,wefindthebesttalentforourclients.AsapermanentemployeeofPeregrine,with accesstoallourstandardbenefits,youwillbedeployedacrossourportfolioofclientsasaspecialistconsultant,workingonawidearrayofcomplexprojectsacrossmultipleindustries. TheRole: AsaDataPlatformEngineerinahighlyregulatedenvironment,youwillberesponsiblefordesigning,building,andmaintainingsecureandscalabledatainfrastructurethatsupportsbothcloudandonpremisesplatforms.Youwillplayakeyroleinensuringthatalldatasystemscomplywithindustryregulationsandsecuritystandardswhileenablingefficientaccessforanalyticsandoperationalteams. AstrongcommandofApacheNiFiisessentialforthisrole.Youwillbeexpectedtodesign, implement,andmaintaindataflowsusingNiFi,ensuringaccurate,efficient,andsecuredata ingestion,transformation,anddelivery.Youshouldbeadeptatidentifyingandresolvingissues withinNiFiflows,managingperformancebottlenecks,andimplementingrobusterrorhandling strategies. Youllworkcloselywithcross-functionalteamsincludingdataarchitects,complianceofficers,and cybersecurityspecialiststointegratedatafromvarioussystemssuchasdatabases More ❯

As an FPGA/Digital Design Engineer, you will: Architect, design, implement, verify, and integrate digital logic (RTL) for FPGA/SoC platforms Work on signal processing, communications, data pipelines, digital beamforming, or related SDR blocks Develop testbench environments, simulations, and formal/verifiable test strategies Perform timing closure, clock domain crossing, resource balancing, power optimisation Define interfaces More ❯

the development and implementation of robust Lab/digital infrastructure within our R&D laboratories for the UK and US locations. This is pivotal in enabling efficient, automated data exchange between laboratory instruments and enterprise informatics systems such as Electronic Lab notebooks (ELN)s, Laboratory Information Management Systems (LIMS) and other relevant data setups. The ideal … implement, and maintain secure, scalable IT/OT interfaces between lab instruments (e.g., bioreactors, plate readers, liquid handlers) and informatics systems (ELN, LIMS, SDMS, etc.) Collaborate with IT, Data science and scientific teams in R&D to identify integration needs and define architecture for real-time or scheduled automated data transfer. Develop or configure middleware, APIs … and custom connectorseither in-house or in collaboration with external vendorsto enable automated data capture, contextualization, and storage. Manage instrument network connectivity (e.g., OPC-UA, MQTT, REST APIs, TCP/IP) and data pipelines from edge devices to cloud/on-premise environments. Support system validation efforts (e.g., CSV, 21 CFR Part 11 compliance) in partnership More ❯

to humanity's enduring challenges. We are looking for a Senior DevSecOps Engineer to join the Pathogen Programme at EIT. In this role, you'll help ensure our data platform is built to the highest standards, with a strong emphasis on automation across the development lifecycle. You'll work closely with engineers to deploy data pipelines … their workflows. You'll be responsible for maintaining infrastructure, designing secure automation pipelines, managing cloud environments, and ensuring security and compliance. You'll collaborate with cross-functional teams, data engineers, backend, and full-stack developers, to build robust, automated deployment pipelines across our environments. Key Responsibilities Design, implement, and maintain secure cloud infrastructure using Oracle Cloud Infrastructure (OCI … tools like Terraform to enable secure, repeatable deployments. Implement and manage CI/CD pipelines, focusing on automated security testing, deployment, and monitoring. Ensure all aspects of the data platform OCI infrastructure, data ingest pipelines, tool deployments, access controls, and monitoring are developed, tested, and deployed using automation best practices. Support bioinformaticians in building pipelines that More ❯

Why Join? Here, your work means more: Make an impact by shaping AI security frameworks for innovative enterprise projects. Be part of challenging, future-defining programmes in AI, data, and cloud security. Collaborate with industry experts and multidisciplinary teams , while driving standards in one of the most rapidly evolving technology landscapes. The Role As an AI Security Architect … will: Define and implement AI/ML security strategies, standards, and guidelines . Design and own enterprise security architecture for AI/ML systems, CI/CD, and data pipelines. Conduct security risk assessments and threat modelling for AI/ML workloads. Support testing of AI/ML models, including trustworthiness, fairness, bias, and adversarial attack detection. Collaborate … with data scientists, engineers, and stakeholders to embed security controls in AI/ML pipelines. Enforce robust access control, authentication, and encryption standards . Stay ahead of emerging AI security threats, regulations, and frameworks . Provide advisory guidance to cross-functional teams during AI/ML system design and deployment. Contribute to regulatory compliance efforts (EU AI Act More ❯

We are a high-growth scale-up building a cloud SaaS platform that enables industrial and energy clients to manage fleets of autonomous systems and turn vast inspection data into actionable intelligence. Our platform is becoming the mission control for critical infrastructure monitoring, combining dashboards, anomaly detection, rules/alerts, and predictive analytics into one solution. Already working … and architecture of a SaaS platform Deliver across backend (Java) and frontend (Angular) systems Build secure, scalable cloud infrastructure (AWS) Implement APIs, authentication (SSO/LDAP), and secure data pipelines Develop dashboards and data visualisations to transform complex datasets into actionable insights Partner with leadership to define the technical vision and roadmap Scale into a leadership … SaaS products Hands-on expertise with Java (backend) and Angular (frontend) Skilled in AWS cloud services Solid understanding of APIs, authentication, and security protocols (SSO, LDAP) Experience with data visualisation and analytics platforms Proven ability to design and deliver scalable, secure architectures Entrepreneurial mindset: comfortable working in fast-paced, scale-up environments Nice-to-Haves Knowledge of machine More ❯

We are a high-growth scale-up building a cloud SaaS platform that enables industrial and energy clients to manage fleets of autonomous systems and turn vast inspection data into actionable intelligence. Our platform is becoming the “mission control” for critical infrastructure monitoring, combining dashboards, anomaly detection, rules/alerts, and predictive analytics into one solution. Already working … and architecture of a SaaS platform Deliver across backend (Java) and frontend (Angular) systems Build secure, scalable cloud infrastructure (AWS) Implement APIs, authentication (SSO/LDAP), and secure data pipelines Develop dashboards and data visualisations to transform complex datasets into actionable insights Partner with leadership to define the technical vision and roadmap Scale into a leadership … SaaS products Hands-on expertise with Java (backend) and Angular (frontend) Skilled in AWS cloud services Solid understanding of APIs, authentication, and security protocols (SSO, LDAP) Experience with data visualisation and analytics platforms Proven ability to design and deliver scalable, secure architectures Entrepreneurial mindset: comfortable working in fast-paced, scale-up environments Nice-to-Haves Knowledge of machine More ❯

active DV Clearance will be preferred. In this role, you'll: Contribute to the design, deployment, and maintainenance of the Splunk environments, including clustered architectures, for large-scale data ingestion and analysis. Reviewing and improving Data collection and Data normalisation Continuous improvement and updating of processes and workflow requirements Working with stakeholders, IT operations … issues. We're looking for someone who can bring: A background in solving complex security challenges. Strong knowledge of SIEM tools, ideally Splunk and Sentinel An understanding of data pipelines, modelling, integrety and compliance. Hands-on experience with Python and other scripting languagues would be an added advantage Flexibility to work onsite and ad hoc travel to London More ❯