1 to 25 of 29 Big Data Jobs in the Thames Valley

Job Title: Senior Data Engineer Location: London, UK (3 days in the office) SC Cleared: Required Job Type: Full-Time Experience: 8+ years Job Summary : We are seeking a highly skilled and experienced Senior Data Engineer to join our team and contribute to the development and maintenance of our cutting-edge Azure Databricks platform for economic data. This … platform is critical for our Monetary Analysis, Forecasting, and Modelling activities. The Senior Data Engineer will be responsible for building and optimising data pipelines, implementing data transformations, and ensuring data quality and reliability. This role requires a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working … with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation: Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data providers) into the Azure Databricks platform. Optimise data pipelines for performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules More ❯

Are you passionate about Data and Analytics (D&A) and excited about how it can completely transform the way an enterprise works? Do you have the strategic vision, technical expertise, and leadership skills to drive data-driven solutions? Do you want to work in a dynamic, fast-growing category? If so, you might be the ideal candidate for … the role in the Data and Analytics function for Global Pet Nutrition (PN) at Mars. Pet Nutrition (PN) is the most vibrant category in the FMCG sector. As we work to transform this exciting category, a new program, Digital First, has been mobilized by the Mars Pet Nutrition (PN) leadership team. Digital First places pet parents at the centre … to demonstrate significant thought leadership, quality decision making, deep technical know-how, and an ability to navigate complex business challenges while building and leading a team of world class data and analytics leaders. With Digital First, PN is moving to a Product based model to create business facing digital capabilities. Develop and maintain robust data pipelines and storage More ❯

Join us as a Data Platform Engineer for our client. At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector – we are always in search of the best talent … benefits, you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries. The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and on-premises … platforms. You will play a key role in ensuring that all data systems comply with industry regulations and security standards while enabling efficient access for analytics and operational teams. A strong command of Apache NiFi is essential for this role. You will be expected to design, implement, and maintain data flows using NiFi, ensuring accurate, efficient, and secure More ❯

Join us as a Data Platform Engineer for our client. At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector – we are always in search of the best talent … benefits, you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries. The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and on-premises … platforms. You will play a key role in ensuring that all data systems comply with industry regulations and security standards while enabling efficient access for analytics and operational teams. A strong command of Apache NiFi is essential for this role. You will be expected to design, implement, and maintain data flows using NiFi, ensuring accurate, efficient, and secure More ❯

Contract Opportunity: Data Engineer - 6-Month Inside IR35 Location: Milton Keynes (Hybrid - 3 Days in Office per Week) Duration: 6 Months IR35 Status: Inside IR35 Rate: Flexible, Depending on Experience We're looking for an experienced Data Engineer to join a project-focused team working on the development and implementation of a robust data model for banking … The engagement is inside IR35 , and rates are flexible based on experience. Role Purpose: You'll collaborate with a small technical team and liaise with IT to ensure the data model is effectively productionised. The ideal candidate will have strong technical data analysis skills, experience with big data technologies, and the ability to communicate complex insights … clearly to stakeholders. Key Responsibilities: Collaborate with a small team to build and refine a banking-focused data model. Liaise with IT teams to transition data models into production environments. Conduct data mining and exploratory data analysis to support model development. Apply strong SQL, Hadoop, and cloud-based data processing skills to manage and analyse More ❯

Responsibilities Designing, developing, and maintaining high-performance, scalable, and secure backend services and APIs using technologies such as Python and NodeJS. Collaborating with science teams, full stack engineers, and data engineers to understand data requirements and integrate user needs into backend services. Building and maintaining RESTful APIs to surface data in the platform, ensuring it is findable … and accessible in line with FAIR principles. Developing APIs that support future interoperability and federation for pathogen diagnostics, epidemiology monitoring, and research. Working with data engineers to build pipelines and processes that store and transform platform data for easy consumption. Implementing and managing data storage solutions including data lakes, data warehouses, and databases such as … object storage, Oracle Autonomous Database, and PostgreSQL. Supporting innovation efforts by exploring new technologies such as vector databases to enable search and AI use cases. Using big data technologies like Kafka, Iceberg, and Parquet, along with managed databases including PostgreSQL and Oracle vector databases. Operating, monitoring, and maintaining Oracle Cloud infrastructure to ensure backend services are highly available More ❯

Exp: 10 + yrs Job Summary The Senior Test Specialist/Architect in Cloud Automation Testing focuses on automating testing processes in ETL (Extract, Transform, Load) and Data Warehousing environments to ensure the quality and reliability of data pipelines. The role involves designing, developing, and implementing automation scripts for testing data transformations, data loading processes, and … data quality verification. Key Responsibilities 1. Develop and execute automated test scripts for etl processes and data pipelines. 2. Collaborate with cross functional teams to design and implement automated testing frameworks. 3. Create test plans and test cases for etl and data warehouse testing. 4. Identify and troubleshoot issues in data transformations and data loading … processes. 5. Conduct performance testing and ensure scalability of data pipelines. 6. Implement best practices for data quality assurance in etl environments. Skill Requirements 1. Proficiency in cloud-based automation testing tools like selenium, appium, or similar. 2. Experience in automation testing of etl (extract, transform, load) processes and data warehousing. 3. Strong understanding of sql for More ❯

GCP Data Engineer | London | £50,000 - £70,000 Peaple Talent have partnered with a specialist data consultancy delivering services across data engineering, data strategy, data migration, BI & analytics. With a diverse portfolio of clients and cutting-edge projects, our client is a trusted name in the data consulting space. Due to exciting growth plans … we are now looking for a Data Engineer , specialising in Google Cloud Platform (GCP) and BigQuery . We are looking for: Demonstrable data engineering/BI skills, with a focus on having delivered solutions in Google Cloud Platform (GCP) Strong experience designing and delivering data solutions using BigQuery Proficient in SQL and Python Experience working with Big Data technologies such as Apache Spark or PySpark Excellent communication skills, with the ability to engage effectively with senior stakeholders Nice to haves: GCP Data Engineering certifications BigQuery or other GCP tool certifications What’s in it for you: 📍 Location: London (Hybrid) 💻 Remote working: Occasional office visits each month 💰 Salary: £50,000–£70,000 DOE 🤝 Collaborative culture More ❯

Databricks Data Engineer | London | £50,000 - £70,000 🧱 Peaple Talent have partnered with a specialist data consultancy delivering services across data engineering, data strategy, data migration, BI & analytics. As a trusted Microsoft Partner, my client are leaders in the industry, with a diverse portfolio of clients and projects. Due to exciting growth plans we are … now looking for a Data Engineer, specialising in Databricks. We are looking for: Demonstratable data engineering/BI skills, with a focus on having delivered in Microsoft Azure Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication … skills, effectively participating with Senior Stakeholders Nice to haves: Azure Data Engineering certifications Databricks certifications What's in it for you: 📍Location: London (Hybrid) 💻Remote working: Occasional office visits each month 💰Salary: £50,000-£70,000 DOE 🤝Collaborative culture and great team support 📈Vast L&D opportunities both internally and externally More ❯

Energy Domain is a must. Under 7 years of experience only. Permanent. Key Responsibilities Design, develop, and maintain data ingestion pipelines using open-source frameworks and tools Build and optimise ETL/ELT processes to handle small to large-scale data processing requirements Develop data models and schemas that support analytics, business intelligence and product needs Monitor … troubleshoot, and optimise data pipeline performance and reliability Collaborate with stakeholders, analysts and product team to understand data requirements Implement data quality checks and validation processes to ensure data integrity Participate in architecture decisions and contribute to technical roadmap planning Technical Skills: Great SQL skills with experience in complex query optimization Strong Python programming skills with … experience in data processing libraries (pandas, NumPy, Apache Spark) Hands-on experience building and maintaining data ingestion pipelines Proven track record of optimising queries, code, and system performance Experience with open-source data processing frameworks (Apache Spark, Apache Kafka, Apache Airflow) Knowledge of distributed computing concepts and big data technologies Experience with version control systems More ❯

Data Analyst – £285/day | 10-Month Contract | Hybrid (London) We’re looking for an experienced Data Analyst to join a global Customer Success Data Science team , working on initiatives that enhance customer experience and operational excellence across international markets. This is an exciting opportunity to make a real impact by building data foundations, enabling actionable … insights, and driving automation across multiple regions. You’ll collaborate closely with cross-functional teams to ensure data drives strategy, informs decision-making, and helps deliver exceptional customer outcomes. What You’ll Do Build strong data foundations: Develop pipelines, improve data integrity, and establish clear metric definitions and governance across multiple international markets. Automate and streamline reporting … operational inefficiencies, turning insights into actionable recommendations. Support experimentation: Analyse A/B tests and digital experience initiatives to measure impact and inform continuous improvement. Tell the story with data: Communicate insights effectively to both technical and non-technical audiences, influencing strategy and driving better decisions. What We’re Looking For 8+ years of experience in Data Analytics More ❯

About Us: Quantiphi is an award-winning Applied AI and Big Data software and services company, driven by a deep desire to solve transformational problems at the heart of businesses. Our signature approach combines groundbreaking machine-learning research with disciplined cloud and data-engineering practices to create breakthrough impact at unprecedented speed. Company Highlights: Quantiphi, a Services … AI Platforms company, has seen 2.5x growth YoY since its inception in 2013 Headquartered in Boston, with 3300+ data science professionals across 11 global offices Winner of 3X NVIDIA AI Partner of the year award Winner of the 17X Google Cloud Partner of the Year award including "Machine Learning, Breakthrough and Social Impact partner Winner of 3X AWS AI … Recognized Leaders by Gartner, Forrester, IDC, ISG, Everest Group and other leading analyst and independent research firms. Recognized in the Fast Company 2021 World Changing Ideas- AI and Data category Winner of 2021 AWS Canada Rising Star of the Year Be part of a trailblazing team that’s shaping the future of AI, ML, and cloud innovation. Your next More ❯

About Tredence: Tredence is a global analytics services and solutions company. Our capabilities range from Data Visualization, Data Management to Advanced analytics, Big Data and Machine Learning. Our uniqueness is in bringing the right mix of technology and business analytics to create sustainable white-box solutions that are transitioned to our clients at the end of … the engagement. We do this cost effectively using a global execution model leveraging our clients' existing technology and data assets. We also come in with strong IP and pre-built analytics solutions in data mining, business intelligence and Big Data. Role Overview As a Client Partner, you’ll work hand-in-hand with senior leaders to nurture … and create lasting impact by helping our clients achieve their business goals You will act as the bridge between business, delivery, and solution teams — ensuring that advanced AI and data-driven solutions translate into measurable business impact for global retail clients. Key Responsibilities Client Relationship Management Serve as the primary point of contact for day-to-day client interactions. More ❯

systems to support Vodafone's secure SOC. Provide technical support to SOC analysts and management of SIEM systems, databases and log/event collection systems. Other duties include: Manage Big Data SIEM Platforms, based on Elastic/Cloudera technologies Implement Log/Event Collection systems and agents for AWS, Azure, GCP and on-premise platforms Support customer deployments … of working as an IT System Administrator/Engineer, ideally in an telecoms/ISP environment Knowledge of a wide range of security technologies, including SIEM platforms Database/Big Data Experience, e.g. MySQL, Postgres, ElasticSearch, Solr, Hadoop IT Systems – Windows, Linux, VMWare, AWS/Azure Scripting – Python, Powershell Worried that you don't meet all the desired More ❯

We're the world's leading data, insights, and consulting company; we shape the brands of tomorrow by better understanding people everywhere. Kantar's Profiles division is home to the world's largest audience network. With access to 170m+ people in over 100 global markets, we offer unrivalled global reach with local relevancy. Validated by industry leading anti-fraud … technology, Kantar's Profiles Audience Network delivers the most meaningful data with consistency, accuracy, and accountability – all at speed and scale. Job Details Location: Reading or London, UK Please note: You must have the right to work in the UK. We are unable to offer visa sponsorship for this role. At Kantar, we help the world's leading brands … make better decisions through data. We're now looking for an experienced Lead Data Engineer to join our team and help us elevate the quality and integrity of our global consumer panels. These panels—more than 50 worldwide—are the foundation of our data-driven insights. Your mission? To build and evolve the data ecosystem, pipelines and More ❯

week 6-12 months Umbrella Only Job description: Design, develop, and maintain ETL solutions using Ab Initio (Graphical Development Environment, Co>Operating System, EME, and Conduct>It). Integrate data from multiple heterogeneous sources into unified, high-quality datasets. Perform performance tuning, debugging, and optimization of existing ETL processes. Implement data quality, validation, and error-handling frameworks. Collaborate … with architects and DBAs on data modeling, pipeline design, and environment setup. Participate in code reviews, technical documentation, and adherence to best practices. Work in Agile/Scrum environments, supporting release planning and production deployment. Mentor junior developers and provide technical guidance to the wider data team. Ensure compliance with data governance and security standards. Strong knowledge … of Ab Initio components - Graphs, Plans, Continuous Flows, Conduct>It, Express>It, Metadata Hub, and PDL. Hands-on experience with SQL , Unix/Linux shell scripting , and data warehouse concepts . Familiarity with big data ecosystems (Hadoop, Hive, Spark) and cloud platforms (AWS, Azure, GCP) is a plus. Proven ability to troubleshoot complex ETL jobs and resolve More ❯

Role: Senior AI/ML Scientist – Personalization & GenAI (Dubai based) Join a high-performing Data Science team whose mission is to drive competitive value through scalable AI solutions. The team builds models that enhance user experiences, enable better decision-making, improve operational efficiency, and shape the regional AI ecosystem. As a senior technical leader, you’ll help define the … Lead end-to-end AI transformation focused on personalization in a consumer-facing application. Define and execute a long-term vision for customer acquisition and engagement strategies powered by data and AI. Conduct exploratory analysis to understand user behavior, discover optimization opportunities, and develop behavior models to inform product enhancements. Design and deploy data/ML instrumentation to … extract insights and optimize the product experience. Provide strategic product guidance through data-driven recommendations, experimentation insights, and root cause analyses. Build and scale machine learning algorithms and pipelines to production using big data technologies. Develop and deploy retrieval-augmented generation (RAG) systems and LLM-based applications. Design and evaluate A/B tests and communicate results More ❯

and engineering problems Experience with MLOps tools and practices, including Airflow, MLflow, and containerization (e.g., Docker) A passion for gaining insight into real-world datasets and clearly communicating through data visualization techniques Interest in material discovery, computer vision, handling big data and optimisation techniques Highly effective communicator who encourages innovation through collaboration Natural problem-solver with a More ❯

and engineering problems Experience with MLOps tools and practices, including Airflow, MLflow, and containerization (e.g., Docker) A passion for gaining insight into real-world datasets and clearly communicating through data visualization techniques Interest in material discovery, computer vision, handling big data and optimisation techniques Highly effective communicator who encourages innovation through collaboration Natural problem-solver with a More ❯

and supporting ML models in production Excellent communicator across technical teams and stakeholders Effective problem-solver who takes initiative in complex production settings Experience with scientific computing, deep learning, big data, or health IT ontologies (e.g., PyTorch, JAX, Spark, HL7, FHIR) (desirable) Familiarity with cloud infrastructure (Azure/AWS), infrastructure as code, Kubernetes, Linux, Docker, data pipelines More ❯

and supporting ML models in production Excellent communicator across technical teams and stakeholders Effective problem-solver who takes initiative in complex production settings Experience with scientific computing, deep learning, big data, or health IT ontologies (e.g., PyTorch, JAX, Spark, HL7, FHIR) (desirable) Familiarity with cloud infrastructure (Azure/AWS), infrastructure as code, Kubernetes, Linux, Docker, data pipelines More ❯

services focus on two core themes: Unlocking Potential Technology Strategy/Architecture/Programme Mobilisation We believe that technology done well can transform an organisation. Whether through digital transformation, data exploitation, artificial intelligence or collaboration and mobility tools, technology has the power to enhance customer experiences, generate insights and improve how organisations work. Working in partnership with business leaders … within a management or IT consulting environment Degree educated (STEM discipline preferred) Broad knowledge of corporate IT infrastructure , ideally across: International corporate networks End-user compute within corporate environments Data centre infrastructure (server, storage, virtualisation) Public cloud platforms (Azure, AWS, GCP) Big data, visualisation and analytics Leadership experience in project and programme delivery Experience as a client More ❯

Assess the practicality and relevance of modern technologies such as machine learning, big data analytics, generative AI, multi-agent systems, and quantum computing in insurance and risk consulting. Work with diverse teams to pinpoint challenges in the industry and devise innovative solutions using these technologies. Examine intricate data sets to offer insights that aid in the strategic More ❯

this firm does, tech is. Our client is an innovative tech driven trading (Equities and ETF’s) company with a pioneering new concept for the financial markets with their big data technology platform. They have assembled a top team from across the financial markets and are looking to expand to meet high demand from clients. Their trading capital … candidate must be: Experienced in Core Java Highly motivated Delivery focused and accountable. Ideally experienced in server side multi-threaded Java – if you haven’t been siloed in a big firm then don’t worry. Additional exposure to the following is desired : - Tech Stack you will learn Hadoop and Flink RUST, Javascript, React, Redux, Flow Linux, Jenkins Kafka, Avro More ❯

this firm does, tech is. Our client is an innovative tech driven trading (Equities and ETF’s) company with a pioneering new concept for the financial markets with their big data technology platform. They have assembled a top team from across the financial markets and are looking to expand to meet high demand from clients. Their trading capital … candidate must be: Experienced in Core Java Highly motivated Delivery focused and accountable. Ideally experienced in server side multi-threaded Java – if you haven’t been siloed in a big firm then don’t worry. Additional exposure to the following is desired : - Tech Stack you will learn Hadoop and Flink RUST, Javascript, React, Redux, Flow Linux, Jenkins Kafka, Avro More ❯