23 of 23 Data Pipeline Jobs in the Thames Valley

Join us as a Data Platform Engineer for our client. At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector – we are always in search of the best … you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries. The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and on … with over 3 years of relevant experience in data engineering, platform engineering, or a related field, with demonstrated hands-on expertise in NiFi and data pipeline design in regulated environments. Responsibilities: Design, develop, and maintain robust and secure data pipelines using NiFi and related big data technologies. Troubleshoot and optimize NiFi More ❯

Data Engineer Salary: Up to £75,000 I am working with a forward-thinking organisation that is modernising its data platform to support scalable analytics and business intelligence across the Group. With a strong focus on Microsoft technologies and cloud-first architecture, they are looking to bring on a Data Engineer to help design … and deliver impactful data solutions using Azure. This is a hands-on role where you will work across the full data stack, collaborating with architects, analysts, and stakeholders to build a future-ready platform that drives insight and decision-making. In this role, you will be responsible for: Building and managing data pipelines using … Azure Data Factory and related services. Building and maintaining data lakes, data warehouses, and ETL/ELT processes. Designing scalable data solutions and models for reporting in Power BI. Supporting data migration from legacy systems into the new platform. Ensuring data models are optimised for performance and reusability. More ❯

Data Engineer Salary: Up to £75,000 I am working with a forward-thinking organisation that is modernising its data platform to support scalable analytics and business intelligence across the Group. With a strong focus on Microsoft technologies and cloud-first architecture, they are looking to bring on a Data Engineer to help design … and deliver impactful data solutions using Azure. This is a hands-on role where you will work across the full data stack, collaborating with architects, analysts, and stakeholders to build a future-ready platform that drives insight and decision-making. In this role, you will be responsible for: Building and managing data pipelines using … Azure Data Factory and related services. Building and maintaining data lakes, data warehouses, and ETL/ELT processes. Designing scalable data solutions and models for reporting in Power BI. Supporting data migration from legacy systems into the new platform. Ensuring data models are optimised for performance and reusability. More ❯

Coeo are trusted data management and analytics experts, delivering technology strategy and support for business. The team have deep technical and commercial experience working with Microsoft Data Services to help our clients optimise their costs and maximise the benefits from their investments in these technologies. Coeo have a strong emphasis on consulting skills, and we expect … been established for over 17 years and has exclusively focused on Microsoft technologies. Our mission is to “ help our clients predict their future ” through the better use of data, technology, people and processes. To do this our business has always focused on Managed Services Data Platform Consultancy Analytics Consultancy Adoption and Change Management There has never … data solutions. Optimize database performance and troubleshoot any issues related to data warehousing, ETL and reporting tools. Provide ongoing support for the data pipeline and reporting systems, ensuring high availability and performance. Continuously evaluate and implement best practices for data management, reporting, and analytics. Required Skills and Qualifications: In-depth knowledge More ❯

Senior Data Engineer (Databricks) Full Time/Permanent £70,000 - £80,000 + car allowance, generous bonus, private medical and other extensive benefits Hybrid - 1 day a week in the Oxfordshire head office required The Company: My client is an industry leading and award-winning financial services organisation who operate on a global scale. They are headquartered in … Oxfordshire, UK and also have offices throughout the UK, Europe and the US. The Role: We are looking for a driven and experienced Senior Data Engineer with extensive Databricks experience to join a rapidly growing in house Data team during an exciting period of change. This is a role where you can really make your mark … an Agile delivery environment, working closely with the Head of Data, Data Product Manager and Data Architect to ensure your team maintains a pipeline of delivery against the Backlog; providing vital insight from a wide-ranging dataset to support executive and operational decision making that will underpin sustained growth of business units domestically More ❯

Your New Role We're looking for an experienced Principal Data Engineer to lead the development and evolution of a modern enterprise data platform. This strategic, hands-on position involves architecting and optimising scalable data pipelines to support advanced analytics, AI/ML initiatives, and actionable insights across the organisation.You'll take full ownership … Acting as the technical authority, you'll define best practices, establish governance frameworks, and mentor engineers and analysts to maximise platform value.This is an opportunity to shape the data landscape and deliver solutions that empower decision-making and innovation. Key Responsibilities Platform Leadership: Design, implement, and manage Snowflake as the … enterprise data hub, ensuring scalability, security, and performance. Data Architecture & Strategy: Define frameworks for ingestion, replication, storage, and transformation across diverse data sources. Pipeline Development: Build efficient ELT pipelines using tools such as DBT and Python, integrating operational, financial, and network data. Performance Optimisation: Configure Snowflake warehouses and partitioning strategies for cost efficiency More ❯

accommodate the delivery of our expanding workload in support of the broader growth of the business. Linked to the steady growth of the business, we are evolving our data architecture to offer a modern data service with new data sources coming online and redeveloping our data warehouse to align with our projected … growth. The current team is small so the role will have exposure to colleagues across Ridge, representing the Data and Analytics team in conversations across the entire business' data estate. Therefore, the role has a broad scope of responsibility, requiring the individual to own projects themselves and deliver end-to-end solutions, with support and discussion … shared across the team. Responsibilities: Integrating new data sources into the data warehouse, and building in best-practice warehouse design Staging & shaping data from multiple data sources into reporting and analytical schemas Experience with data warehouse modelling, designed to accommodate future growth Creation of technical documentation, capturing development, processes & DW More ❯

Data Architect (SQL Azure) Milton Keynes/WFH to £90k Are you a Data technologist with data architecture experience looking for an opportunity to take ownership and make an impact? You could be progressing your career at a well known household name within the fast food space as they invest heavily in their data strategy. As the Data Architect you'll be instrumental in delivering a modern cloud based data infrastructure, they have 16 million customer data points across the UK and Ireland that they're looking to use this to optimise customer retention, product quality and speedy and efficient delivery to the customer. Initially you … ll be responsible for migrating the current on-premise data platform to Azure using Databricks; you'll collaborate with the Head of Digital and AI to modernise the current Data infrastructure and monetise the data available, working with senior business stakeholders to deliver tangible business value. Location/WFH: There's a hybrid work More ❯

Your New Role We're seeking a Senior Data Engineer to help shape and maintain a modern enterprise data platform. In this role, you'll design and optimise scalable data pipelines and models that bring data from core business systems into Snowflake, enabling analytics, reporting, and data-driven insights across … pipelines and models in Snowflake to support analytics and reporting needs. Architecture Implementation: Apply defined data architecture standards to ingestion, transformation, storage, and optimisation processes. Pipeline Development: Develop robust ELT/ETL workflows using dbt and orchestration tools, ensuring reliability and maintainability. Performance & Cost Optimisation: Configure Snowflake warehouses and implement query optimisation techniques for efficiency. … What You'll Need to Succeed Proven experience delivering cloud-based data engineering solutions, ideally with Snowflake. Strong proficiency in SQL, Python, and dbt for transformations and pipeline automation. Practical experience with Snowflake features and RBAC management. Familiarity with ingestion tools (Airbyte, Fivetran, Hevo) and cloud services (AWS preferred). Solid understanding of data modelling More ❯

Data Engineering Lead - Hybrid - Azure/Databricks - London Permanent Salary: £90,000 Are you a seasoned Data Engineering professional ready to lead the charge in building scalable, high-performance data platforms? An industry leading, global firm is seeking a Data Engineering Lead to drive innovation across their hybrid Azure/Databricks environment. … About the role: Shape data strategy for global projects Lead a talented team of BI Developers Enjoy a flexible working culture and industry-leading benefits Work with cutting-edge tech in a collaborative, inclusive environment Key Responsibilities: Architect and optimize robust data pipelines for large-scale ingestion, transformation, and delivery Leverage Azure and Databricks to enable … seamless data integration and enrichment Implement best practices for orchestration, monitoring, and error handling Lead and mentor a BI team, ensuring timely delivery of dashboards and data products Champion data governance, quality, and security across platforms Requirements: Proven expertise in Azure, SQL, Databricks, and Azure Data Factory Experience implementing medallion architecture frameworks More ❯

Fabric Consultant Location: Hybrid in London Salary: Up to £90,000 + Benefits Are you passionate about data engineering? Join a leading Microsoft partnered consultancy delivering cutting-edge solutions with Azure, Power BI, and Microsoft Fabric. This is your chance to work on exciting projects, design robust data architectures, and help organisations unlock the full potential … of their data. Key Responsibilities: Build and optimise data pipelines and ETL workflows using Microsoft Fabric and Azure Synapse or Databricks. Implement scalable solutions for data ingestion, storage, and transformation. Develop clean, reusable Python code for data engineering tasks. Research and integrate the latest cloud-based technologies. Requirements: Proven experience in data engineering with Azure tools (Synapse, Data Factory, Databricks). Experience with Microsoft Fabric. Strong Python programming skills. Knowledge of data modelling and Kimball methodology. Excellent communication and problem-solving skills. Fabric certifications are highly desirable Benefits: Competitive salary and bonus scheme Private medical cover, life assurance, and pension. Hybrid working. And more More ❯

and maintain backend control software for advanced scientific systems. Collaborate with scientists, hardware, and systems engineers to architect and optimise performance. Develop tools and interfaces to manage experiments, data pipelines, and control sequences. Apply strong software hygiene, testing and CI/CD practices. Contribute to design reviews and mentor junior engineers. Tech Environment Languages: Python (core), Rust (or More ❯

deploying scientific software using common CI and containerization solutions. Enjoy staying up to date with current software development best practices and tools, e.g. AI coding assistants or modern data pipelines and scheduling solutions. Enjoy diving into new problems with a continuous learning attitude, absorbing relevant domain knowledge. Additional experience with scientific C++ and Fortran libraries, HPC technologies, cloud More ❯

to humanity's enduring challenges. We are looking for a Senior DevSecOps Engineer to join the Pathogen Programme at EIT. In this role, you'll help ensure our data platform is built to the highest standards, with a strong emphasis on automation across the development lifecycle. You'll work closely with engineers to deploy data pipelines … their workflows. You'll be responsible for maintaining infrastructure, designing secure automation pipelines, managing cloud environments, and ensuring security and compliance. You'll collaborate with cross-functional teams, data engineers, backend, and full-stack developers, to build robust, automated deployment pipelines across our environments. Key Responsibilities Design, implement, and maintain secure cloud infrastructure using Oracle Cloud Infrastructure (OCI … tools like Terraform to enable secure, repeatable deployments. Implement and manage CI/CD pipelines, focusing on automated security testing, deployment, and monitoring. Ensure all aspects of the data platform OCI infrastructure, data ingest pipelines, tool deployments, access controls, and monitoring are developed, tested, and deployed using automation best practices. Support bioinformaticians in building pipelines that More ❯

Python - ML - LLM Data Pipelines and ETL - API Integration Familiar with .NET technologies Lead AI Engineer Location: Slough (Hybrid, two days per week in the office) Salary: £80,000 to £100,000 plus benefits Type: Full time About the Organisation We are developing an advanced AI driven insight and decision platform used within industrial and manufacturing environments. The … platform connects to maintenance, inventory and procurement systems and is designed to turn complex operational data into predictive insight and automated decision making. Our aim is to help factories and large operational teams move from manual processes to intelligent and automated workflows. The Opportunity We are looking for a Lead AI Engineer to take ownership of the AI … reduce costs and automate routine procurement activity. A key focus of the role is building agent driven ordering systems. This includes developing AI agents that can gather price data from public sources, compare supplier options, validate stock requirements and place orders on behalf of users with the correct safety checks and approval flows. Key Responsibilities Design and build More ❯

DOE An exciting opportunity for a Software Engineer to join a growing tech company developing a cloud platform used to manage autonomous robotic inspections and transform sensor data into actionable insights for industrial clients. The Role: Youll work across backend, frontend, and data services, contributing to new features, improving existing systems, and supporting integrations with robotics … years experience who wants to develop full-stack skills and work with cutting-edge technologies. Key Responsibilities: Develop features across backend and frontend systems Build and maintain APIs, data pipelines, and UI components Write clean, maintainable code following best practices Support cloud deployment (AWS experience beneficial) Build dashboards and tools for processing inspection data Collaborate with … engineering fundamentals Ability to learn new technologies quickly Good communication and teamwork skills Desirable: AWS or cloud deployment familiarity Interest in robotics, embedded systems, or IoT Experience with data visualisation or real-time data Exposure to AI/ML or computer vision More ❯

DOE An exciting opportunity for a Software Engineer to join a growing tech company developing a cloud platform used to manage autonomous robotic inspections and transform sensor data into actionable insights for industrial clients. The Role: You’ll work across backend, frontend, and data services, contributing to new features, improving existing systems, and supporting integrations with … years’ experience who wants to develop full-stack skills and work with cutting-edge technologies. Key Responsibilities: Develop features across backend and frontend systems Build and maintain APIs, data pipelines, and UI components Write clean, maintainable code following best practices Support cloud deployment (AWS experience beneficial) Build dashboards and tools for processing inspection data Collaborate with … engineering fundamentals Ability to learn new technologies quickly Good communication and teamwork skills Desirable: AWS or cloud deployment familiarity Interest in robotics, embedded systems, or IoT Experience with data visualisation or real-time data Exposure to AI/ML or computer vision More ❯

React) Desirable: AWS familiarity We are a growing technology company building a platform that supports fleets of autonomous inspection robots and turns large volumes of sensor and image data into clear, actionable intelligence. Our software helps major industrial and energy organisations monitor critical infrastructure safely and efficiently. This role sits in the team developing the cloud platform and … user interfaces that operators use to run missions, view inspection data, manage alerts, and interact with AI analysis tools. The Opportunity We’re looking for a software engineer with solid foundations and roughly 2–3 years of professional experience. You’ll join a small, experienced team and work across back end, front end, and robotics-related data services. You’ll help deliver new platform features, improve our existing services, and have the chance to broaden your skills across multiple domains: cloud, data processing, UI development, and embedded/robotics integration. This is a role for someone curious, adaptable, and willing to learn. Why Join Us? Work at the intersection of cloud software, robotics More ❯

AI driven insight and decision platform used within industrial and manufacturing environments. The platform connects to maintenance, inventory and procurement systems and is designed to turn complex operational data into predictive insight and automated decision making. Our aim is to help factories and large operational teams move from manual processes to intelligent and automated workflows. The Opportunity We … reduce costs and automate routine procurement activity. A key focus of the role is building agent driven ordering systems. This includes developing AI agents that can gather price data from public sources, compare supplier options, validate stock requirements and place orders on behalf of users with the correct safety checks and approval flows. Key Responsibilities Design and build … AI powered features including intelligent search, agent driven ordering, predictive stock recommendations and automated procurement workflows Develop autonomous agents that can collect price data, assess supplier choices and initiate or complete ordering actions through APIs or connected systems Build retrieval augmented generation pipelines and vector search capabilities to support accurate and grounded decision making Integrate AI functionality with More ❯

We are a high-growth scale-up building a cloud SaaS platform that enables industrial and energy clients to manage fleets of autonomous systems and turn vast inspection data into actionable intelligence. Our platform is becoming the “mission control” for critical infrastructure monitoring, combining dashboards, anomaly detection, rules/alerts, and predictive analytics into one solution. Already working … and architecture of a SaaS platform Deliver across backend (Java) and frontend (Angular) systems Build secure, scalable cloud infrastructure (AWS) Implement APIs, authentication (SSO/LDAP), and secure data pipelines Develop dashboards and data visualisations to transform complex datasets into actionable insights Partner with leadership to define the technical vision and roadmap Scale into a leadership … SaaS products Hands-on expertise with Java (backend) and Angular (frontend) Skilled in AWS cloud services Solid understanding of APIs, authentication, and security protocols (SSO, LDAP) Experience with data visualisation and analytics platforms Proven ability to design and deliver scalable, secure architectures Entrepreneurial mindset: comfortable working in fast-paced, scale-up environments Nice-to-Haves: Knowledge of machine More ❯

WFH Contract Type: Permanent TECHNE are representing exclusively a pioneering Battery Intelligence technology company is redefining how batteries are understood, optimised, and deployed. By combining electrochemistry, software, and data science , they deliver intelligent battery solutions across the full value chain - from automotive and energy storage to heavy-duty applications. They are now seeking a Cloud Application Engineer to … join their growing technical delivery team. This is a 90% technical, 10% customer-facing role focused on building and customising data pipelines , integrating battery analytics into customer frameworks, and collaborating closely with the product and cloud development teams. Role Overview You will act as the primary technical contact on customer-facing projects, translating requirements and providing technical support … and the core cloud development team. The role involves building automated customer pipelines for battery analytics , customising the existing codebase for customer-specific orchestration frameworks, and turning raw data into actionable, business-relevant insights. You’ll also contribute to the ongoing development of the Battery Intelligence Cloud Platform , helping to accelerate global electrification by improving the performance, safety More ❯

enables real-time biological analysis at scale. This is your opportunity to work at the intersection of science and software , building the systems behind next-generation sequencing and data platforms from portable lab devices to large-scale research infrastructure used by institutions worldwide. The Role You’ll join a collaborative international engineering team developing and maintaining high-impact … data applications that support critical scientific and commercial operations. You’ll work on systems that process large biological datasets , support telemetry pipelines , and help deliver cutting-edge tools for researchers and scientists around the world. What You’ll Do Design, build and maintain data-intensive web applications and APIs Develop scalable systems using Python (Django, FastAPI … background in building web apps, APIs, or workflow engines Hands-on experience with Docker , CI/CD pipelines , and cloud-native environments Knowledge of databases (SQL, MongoDB) and data pipelines Familiarity with Kubernetes , Terraform , or related orchestration tools What’s on offer: Competitive salary and bonus Private healthcare & pension Collaborative, mission-driven culture If you’re a Python More ❯

looking for a hands-on Lead Software Engineer to own the design and delivery of our IoT and cloud software platform. Youll connect our ultra-fast chargers, manage data and remote control functions, and ensure security and scalability across our systems. Youll work directly with the VP Product and VP Power Electronics, shaping architecture, integrations and performance turning … ideas into real, working solutions. ________________________________________ What Youll Do Build IoT and cloud software for charger connectivity and monitoring Create telemetry and data pipelines (PLC, CAN) Enable secure remote control and bi-directional communication (MQTT/WebSockets) Deliver OTA firmware updates and REST APIs (OCPP/OCPI) Develop secure, scalable cloud infrastructure (AWS IoT, Azure, etc.) Collaborate closely with More ❯