1 to 25 of 30 Data Pipeline Jobs in the Thames Valley

Engineer, you will operate within an Agile delivery environment, working closely with the Data Product Manager and Data Architect to ensure your team maintains a pipeline of delivery against the Backlog; providing vital insight from our wide-ranging dataset to support executive and operational decision making that will underpin sustained growth of BFS business units … solutions. You will also have management responsibilities for a small team of Data Engineers, who you will coach, support and organise to ensure we sustain a predictable pipeline of delivery, whilst ensuring all appropriate governance and best practice is adhered to. Your responsibilities as our Senior Databricks Data Engineer will include: Understand the business/… Azure DevOps, collaborating on Sprint cycles, code deployment, version control, and development practices Develop and maintain data models, schemas, and documentation Monitor and troubleshoot data pipeline issues, ensuring timely resolution Stay up-to-date with the latest industry trends and technologies in data engineering and recommend improvements to existing systems or processes as More ❯

Location: Worcester Duration: 6 month initial contract Rate: (Outside IR35) Security: Active DV clearance required Role details: We are looking for 3 x Data Engineers to join our defence & security client on a contract basis. You will helpdesign, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi … These roles are supporting our clients team in Worcester (fully onsite), and requires active UK DV clearance. Key Responsibilities: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi Implement data ingestion, transformation, and integration processes, ensuring data quality and security Collaborate with data … cross-functional teams. Detail-oriented with a focus on ensuring data accuracy, quality, and security. Proactive problem-solving mindset and ability to troubleshoot complex data pipeline issues Desirable skills: Experience working in government, defence, or highly regulated industries with knowledge of relevant standards. Experience with additional data processing and ETL tools like Apache More ❯

Location: Worcester Duration: 6 month initial contract Rate: (Outside IR35) Security: Active DV clearance required Role details: We are looking for 3 x Data Engineers to join our defence & security client on a contract basis. You will helpdesign, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi … These roles are supporting our clients team in Worcester (fully onsite), and requires active UK DV clearance. Key Responsibilities: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi Implement data ingestion, transformation, and integration processes, ensuring data quality and security Collaborate with data … cross-functional teams. Detail-oriented with a focus on ensuring data accuracy, quality, and security. Proactive problem-solving mindset and ability to troubleshoot complex data pipeline issues Desirable skills: Experience working in government, defence, or highly regulated industries with knowledge of relevant standards. Experience with additional data processing and ETL tools like Apache More ❯

Location: Worcester Duration: 6 month initial contract Rate: (Outside IR35) Security: Active DV clearance required Role details: We are looking for 3 x Data Engineers to join our defence & security client on a contract basis. You will helpdesign, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi … These roles are supporting our clients team in Worcester (fully onsite), and requires active UK DV clearance. Key Responsibilities: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi Implement data ingestion, transformation, and integration processes, ensuring data quality and security Collaborate with data … cross-functional teams. Detail-oriented with a focus on ensuring data accuracy, quality, and security. Proactive problem-solving mindset and ability to troubleshoot complex data pipeline issues Desirable skills: Experience working in government, defence, or highly regulated industries with knowledge of relevant standards. Experience with additional data processing and ETL tools like Apache More ❯

Location: Worcester Duration: 6 month initial contract Rate: (Outside IR35) Security: Active DV clearance required Role details: We are looking for 3 x Data Engineers to join our defence & security client on a contract basis. You will helpdesign, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi … These roles are supporting our clients team in Worcester (fully onsite), and requires active UK DV clearance. Key Responsibilities: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi Implement data ingestion, transformation, and integration processes, ensuring data quality and security Collaborate with data … cross-functional teams. Detail-oriented with a focus on ensuring data accuracy, quality, and security. Proactive problem-solving mindset and ability to troubleshoot complex data pipeline issues Desirable skills: Experience working in government, defence, or highly regulated industries with knowledge of relevant standards. Experience with additional data processing and ETL tools like Apache More ❯

Location: Worcester Duration: 6 month initial contract Rate: (Outside IR35) Security: Active DV clearance required Role details: We are looking for 3 x Data Engineers to join our defence & security client on a contract basis. You will helpdesign, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi … These roles are supporting our clients team in Worcester (fully onsite), and requires active UK DV clearance. Key Responsibilities: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi Implement data ingestion, transformation, and integration processes, ensuring data quality and security Collaborate with data … cross-functional teams. Detail-oriented with a focus on ensuring data accuracy, quality, and security. Proactive problem-solving mindset and ability to troubleshoot complex data pipeline issues Desirable skills: Experience working in government, defence, or highly regulated industries with knowledge of relevant standards. Experience with additional data processing and ETL tools like Apache More ❯

and a range of benefits to support your finances, wellbeing and family. Working Pattern: Permanent Full Time Flexible First options available The Role Were looking for a Senior Data Engineer to join the Asset Data & Insights value stream. Youll lead the design, development and operations of robust, scalable data pipelines using modern cloud technologies … empowering real-time analytics, advanced BI, and data-driven decision-making across SSE Thermal. Youll need hands-on expertise with the Databricks ecosystem, a strong command of data engineering best practices, and a growth mindset that balances technical excellence with collaboration and continuous improvement. Experience in an asset-intensive or energy environment is desirable, as is … Delta Live Tables, Unity Catalog) to serve downstream analytics and reporting. - Optimise data architecture for performance, scalability, and reliability, proactively monitoring data health and troubleshooting pipeline issues. - Mentor junior engineers and foster a knowledge-sharing culture through documentation, pairing, and architectural guidance. - Collaborate with analysts, product teams, and business stakeholders to translate requirements into performant More ❯

Data Engineers - Permanent Opportunity Location Reading: 1-2 days per week A major UK organization is expanding their Data Engineering team to drive forward their ambitious data platform transformation initiative. Are you an experienced Data Engineer ready to tackle complex data challenges at scale? This established organization is building their … next-generation Enterprise Data Platform and needs skilled professionals to accelerate their journey toward becoming a data-centric organization. About Our Client Our client operates critical infrastructure serving millions of customers across the UK. They're investing significantly in modernizing their data capabilities, creating a centralized data lakehouse architecture that will transform … Help establish and maintain enterprise-wide data governance standards Core Technical Requirements of the Data Engineer Azure Data Factory mastery - demonstrable expertise in pipeline development and management Proficient SQL and/or Python capabilities - minimum 2-3 years hands-on experience Comprehensive Data Engineering background - proven track record in enterprise data More ❯

Job Description: We are seeking a highly skilled Data Architect for Gen AI Engineering to lead the design and implementation of robust data architectures that support our generative AI initiatives across various business functions, including Marketing, Product Development, and Customer Insights. This role is transformation-funded and therefore has an expected end date of 31st December … 2026. What are we looking for? A seasoned data architect with 8+ years of experience in data architecture and engineering, specifically in the context of generative AI and machine learning applications. Strong expertise in designing scalable data models and architectures, with proficiency in … cloud platforms such as AWS, GCP, or Azure, and familiarity with data warehousing solutions. Experience with data integration tools, ETL processes, and data pipeline development, along with a solid understanding of data governance and security best practices. Proven leadership skills in managing cross-functional teams, driving data strategy, and More ❯

officially designated Posts. Our UK-based staff work in King Charles Street, London, Abercrombie House in East Kilbride and in Milton Keynes. These roles contribute to FCDOs Data Strategy to improve insights to the business by provisioning it with reliable and trustworthy data sources and improving theirdeliveryin a secure manner in compliance with Data regulations. The team is primarily Azure focused but other Cloud technologies such as Oracle andAmazon Web Services (AWS) are usedin FCDO. The successful candidates will build complex data pipelines (both Extract Transform and Load ETL and Extract Load and Transform ELT ) in the Azure cloud platforms. You will work with structured and unstructured data, data lakes, data warehouses to service operational and analytical business needs. The successful candidates will: Work collaboratively with Enterprise Data Colleagues to develop pipelines and data product that aligns with data architecture and engineering best practice. Develop data assets and products that assist Power Business Intelligence (BI) colleagues to More ❯

benefits to support your finances, wellbeing and family. Working Pattern: Permanent Full Time Flexible First options available The Role We are recruiting for the position of a Lead Data Engineer in SSE's Group Data and Analytics team. This team acts as a Centre for Enablement (CfE) for the SSE organisation and are involved in a … variety of activities: practical project development work, advising on best practice, investigation into new capabilities and other activities that further data driven solutions across SSE. You Will - Act as a subject matter expert in the data engineering/data solutions space as a key member of the CfE. - Understand business requirements and discuss with … stakeholders a recommended solution approach. Work in an inclusive manner with team members in terms of architecture and design of a solution. - Lead, mentor and advise the CfE data solutions team in their work. - Lead and undertake practical work as well as provide advice and guidance to business units in SSE. This will mainly relate to data More ❯

Data Engineer Amersham – Hybrid £45,000 - £50,000 VIQU have partnered with a UK retail company who are looking to recruit a Data Engineer to play a critical part in the leveraging of data to drive insights, optimise business performance and support decision making. Key Responsibilities of the Data Engineer: Develop, maintain … and optimise data pipelines and integration workflows across various platforms, including SQL Server and cloud environments. Collaborate with cross-functional teams to gather requirements and deliver tailored data solutions. Build and refine automated data processes ensuring data accuracy and availability for reporting and analysis. Support the rollout of an Azure-based data … Participate in peak reporting cycles and ad hoc data requests as needed. Key Requirements of the Data Engineer: Strong expertise in SQL, SSIS for ETL pipeline creation, and SSRS for report development. Experience with Power BI for data visualisation. Familiarity with Azure data services such as Data Lake, Data More ❯

Contract Data Engineer Financial Services 6+ Months Competitive Day Rate My client, a global consultancy, is looking for an experienced Data Engineer to join on a contract basis to support major digital transformation projects with Tier 1 banks. You'll help design and build scalable, cloud-based data solutions using Databricks , Python , Spark , and … traffic financial applications. Key Skills & Experience: Strong hands-on experience with Databricks , Delta Lake , Spark Structured Streaming , and Unity Catalog Advanced Python/PySpark and big data pipeline development Familiar with event streaming tools ( Kafka , Azure Event Hubs ) Solid understanding of SQL , data modelling , and lakehouse architecture Experience deploying via CI/CD tools (e.g. … Azure DevOps, GitHub Actions) Nice to Have: Knowledge of Scala/Java Understanding of GDPR and handling sensitive data This is a contract role (UK-based) offering the chance to work on high-impact projects shaping the future of finance with some onsite presence required. More ❯

Job Title: Data Analyst - Data Engineering Location: Remote/Reading, UK (Hybrid - Office & Remote) Type: 12 months Fixed-Term Contract (FTC) Our client is a market leader in land and property data solutions across the UK, empowering the property sector with powerful digital tools and data-driven insights. From government bodies and … legal firms to mortgage providers and environmental consultants, their work is transforming how land information is accessed and utilised nationwide. We're now looking for a talented Data Analyst to join their Data Engineering team, contributing to a high-profile national digital transformation programme. If you're passionate about uncovering insights from complex datasets and thrive … to define data needs and deliver insights. Build dashboards and reports using tools like Power BI or Tableau. Support ETL workflows and contribute to data pipeline development. Ensure accuracy and consistency of data throughout transformation. Present findings clearly to both technical and non-technical stakeholders. Play a key role in the HM Land More ❯

innovation and collaboration. We are committed to cultivating a community of excellence through trust, innovation, and resilience, valuing diverse experiences and expertise. Requirements The Role: We seek a Data Engineer to develop and deploy a data platform supporting AI initiatives at EIT. You will be responsible for building scalable data infrastructure and pipelines, collaborating … with multidisciplinary teams. Key Responsibilities: Design and implement scalable data pipelines and infrastructure Develop data processing systems using cloud technologies Optimize data storage and retrieval for AI applications Support AI/ML model development and deployment Evaluate and adopt new data technologies Collaborate with scientists, researchers, and operational teams Qualifications & Experience: Extensive … Python programming experience in enterprise environments Knowledge of data modeling, ETL, data warehousing, and SQL Cloud computing experience Strong communication skills Innovative mindset and ability to solve challenges creatively Ability to work in a fast-paced, startup-like environment Nice to have: Familiarity with AI/ML workflows Experience with Docker/Kubernetes Knowledge of Git More ❯

Git, CI/CD methodologies, and cloud services. MarTech Innovation: Research and implement AI solutions specifically targeting marketing use cases (content generation, audience analytics, personalisation). Data Pipeline Architecture: Design robust data flows between marketing systems and AI services. Technical Leadership: Provide strategic direction, mentorship, and best practices for AI implementation across the team. Documentation … and cloud platforms (GCP, Azure). Skilled in containerisation and orchestration (Docker, Kubernetes), version control (Git, BitBucket), and CI/CD pipelines. Strong foundation in software design patterns, data structures, algorithms, and database technologies (SQL, NoSQL, vector databases). Preferred Qualifications Experience in MarTech, AdTech, or B2B marketing technology, with a background in marketing data and … analytics platforms. Familiarity with prompt engineering, LLM optimisation, and designing data processing pipelines (ETL). Agile development experience (Scrum, Kanban) and a track record of integrating AI into business applications. What you need to do now If you're interested in this role, click 'apply now' to forward an up-to-date copy of your CV, or call More ❯

using Docker, Git, CI/CD, and cloud services. MarTech Innovation: Research and implement AI solutions for marketing use cases like content generation, analytics, and personalisation. Data Pipeline Architecture: Design robust data flows between marketing systems and AI services. Technical Leadership: Provide strategic direction, mentorship, and best practices for AI implementation. Documentation: Create comprehensive technical … and cloud platforms (GCP, Azure). Skilled in containerisation and orchestration (Docker, Kubernetes), version control (Git, BitBucket), and CI/CD pipelines. Strong foundation in software design patterns, data structures, algorithms, and database technologies (SQL, NoSQL, vector databases). Preferred Qualifications Experience in MarTech, AdTech, or B2B marketing technology, with a background in marketing data and … analytics platforms. Familiarity with prompt engineering, LLM optimisation, and designing data processing pipelines (ETL). Agile development experience (Scrum, Kanban) and a track record of integrating AI into business applications. What you need to do now If you're interested in this role, click 'apply now' to forward an up-to-date copy of your CV, or call More ❯

security architecture for AI/ML systems, CI/CD and data pipelines. * Participate in enterprise-wide architecture initiatives for AI/ML. Understand the workflow and pipeline architectures of ML and deep learning workloads. * Conduct security risk assessments and threat modelling for AI/ML and other business projects performed thorough design reviews and security assessments … testing of AI/ML models, LLM and development of trustworthiness, safety, fairness, bias and inappropriateness detection, as well as adversarial test specification, development, and execution. * Collaborate with data scientists, engineers, and other stakeholders to embed security controls and measures into AI/ML applications, models, algorithms, and data pipelines. * Define and enforce robust access controls … authentication mechanisms, and encryption standards to safeguard AI/ML systems, training data, provenance, and any sensitive data. * Stay abreast of emerging AI security threats, trends, and technologies, evaluating their impact and proactively adapting security measures to mitigate potential risks. * Provide technical guidance and support to cross-functional teams during the design, development, and deployment of AI/ More ❯

Join a mission-driven team shaping the future of geospatial data engineering. We’re looking for a Principal Data Engineer to lead the design and delivery of cutting-edge data solutions across cloud platforms and spatial technologies. If you're passionate about geospatial data, cloud architecture, and building scalable pipelines, this is … your chance to make a real impact. What You’ll Be Doing: Architecting and implementing robust data pipelines using FME and QGIS . Leading cloud-based data engineering projects across AWS and Azure . Collaborating with cross-functional teams to deliver high-impact geospatial solutions. Mentoring junior engineers and setting best practices for data engineering excellence. Driving innovation in spatial data processing, automation, and analytics. What You’ll Bring: Proven experience as a Senior or Principal Data Engineer . Deep expertise in QGIS and FME for geospatial data transformation. Strong hands-on experience with AWS (e.g., S3, Lambda, Glue) and/or Azure (e.g., Data More ❯

Head of Enterprise Data Engineering - FCDO - G6 London, Milton Keynes or East Kilbride - You will be required to work from this location in line with FCDO policies on Hybrid Working (minimum 60% in the office). About the job Job summary The Foreign, Commonwealth & Development Office pursues our national interests and projects the UK's role in the … Directorate (IDD) is responsible for delivering the Foreign Secretary's ambition to make the FCDO the most technologically integrated and innovative foreign ministry globally. This means harnessing digital, data, technology, IT security and information management to support the FCDO and UK Government partners on our platform to deliver our diplomatic, development and consular aims. All while keeping our … to global network management, from supporting staff in using the latest Microsoft toolsets, to creating corporate applications, harnessing AI and much more in line with the FCDO Digital, Data and Technology (DDaT) Strategy. The Directorate is made up of a wide range of talented staff and experts including specialist, contractors, home civil servants and diplomats who have served More ❯

best practices, emerging threats, and regulatory requirements. Youll work hands-on and collaboratively with cross-functional teams to embed security into the entire AI/ML lifecycle from data pipelines to model deployment. Key Responsibilities Architect and implement security controls for AI/ML systems, data pipelines, and CI/CD processes. Conduct security risk assessments … and threat modelling on AI/ML workflows. Drive secure coding practices and integrate security into MLOps/DevOps pipelines. Address challenges such as adversarial attacks, data poisoning, and prompt injection. Define access control, authentication, and encryption standards across platforms. Work with data scientists and engineers to embed security into model development. Stay current on evolving … Security Architect with direct focus on AI/ML security. Strong knowledge of AI/ML technologies, frameworks (e.g. TensorFlow, PyTorch), and threat landscapes. Background in development or data science is highly advantageous. Expertise in secure development practices and cloud-native architectures. Hands-on experience with threat modelling, adversarial testing, and CI/CD security. Familiarity with tools More ❯

Job Description: Are you passionate about Data and Analytics (D&A) and excited about how it can transform enterprise operations? Do you possess strategic vision, technical expertise, and leadership skills to drive data-driven solutions? If so, you might be the ideal candidate for the Data and Analytics role within Mars's Global Pet … achieve growth, profitability, and customer centricity. This role requires demonstrating thought leadership, technical proficiency, and the ability to navigate complex challenges while leading a team of top-tier data and analytics professionals. The role involves transitioning to a product-based model to develop digital capabilities. Responsibilities include developing and maintaining robust data pipelines and storage solutions … to support analytics and machine learning initiatives. You will report to the Director of Data Engineering Solutions and collaborate globally with engineering teams across core products. Key Responsibilities: Technical Leadership: Provide leadership to data and DevOps engineers. Collaborate on designing and evolving scalable data platforms. Promote best practices and foster a high-performance culture. More ❯

Job Title : Commercial Data Analyst - SQL Location : Didcot, Oxfordshire (Hybrid - 3 days in office, 2 days remote) Salary : £55,000 + Excellent Benefits Introduction Are you passionate about turning complex data into actionable insights? We have a fantastic opportunity for a skilled Commercial Data Analyst to join an exciting company in the retail sector … in the office, with 2 days from home. The company, a leader in sustainable retail, has been in business for over a decade and is known for leveraging data to drive customer experiences and business growth. Duties & Responsibilities Collaborate with data and tech teams to manage data pipelines across eCommerce platforms using GTM/… supply/demand across regions. Analyze customer acquisition channels, calculating ROI and attribution for various marketing efforts. What Experience is Required? At least 5 years of experience in data analysis with a strong focus on data extraction, analysis, and manipulation. Excellent experience in margins, revenue and pricing. Expertise in GTM/GA4, BigQuery, SQL, and Excel More ❯

JOB DETAILS - 500- 550 PER DAY - OUTSIDE IR35 - REMOTE ROLE - 3-MONTH CONTRACT WITH POTENTIAL FOR EXTENSION - NPPV3 AND SC CLEARANCE REQUIRED SKILLS - Extensive experience in Azure Data Factory, Databricks and Synapse. - Knowledge of Oracle. - Understanding of security protocols, dealing with policing data and clearance requirements. RESPONSIBILITIES - Strong collaboration skills with other teams and colleagues within … the organisation. - Ability to communicate effectively with non-technical and junior colleagues. - Taking a leading role in data transformation, building data pipelines and data modelling. What you need to do now If you're interested in this role, click 'apply now' to forward an up-to-date copy of your CV, or call us More ❯

JOB DETAILS - £500-£550 PER DAY - OUTSIDE IR35 - REMOTE ROLE - 3-MONTH CONTRACT WITH POTENTIAL FOR EXTENSION - NPPV3 AND SC CLEARANCE REQUIRED SKILLS - Extensive experience in Azure Data Factory, Databricks and Synapse. - Knowledge of Oracle. - Understanding of security protocols, dealing with policing data and clearance requirements. RESPONSIBILITIES - Strong collaboration skills with other teams and colleagues within … the organisation. - Ability to communicate effectively with non-technical and junior colleagues. - Taking a leading role in data transformation, building data pipelines and data modelling. What you need to do now If you're interested in this role, click 'apply now' to forward an up-to-date copy of your CV, or call us More ❯