1 to 25 of 42 Big Data Jobs in the Thames Valley

Job Title: Senior Data Engineer Location: London, UK (3 days in the office) SC Cleared: Required Job Type: Full-Time Experience: 8+ years Job Summary : We are seeking a highly skilled and experienced Senior Data Engineer to join our team and contribute to the development and maintenance of our cutting-edge Azure Databricks platform for economic data. This … platform is critical for our Monetary Analysis, Forecasting, and Modelling activities. The Senior Data Engineer will be responsible for building and optimising data pipelines, implementing data transformations, and ensuring data quality and reliability. This role requires a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working … with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation: Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data providers) into the Azure Databricks platform. Optimise data pipelines for performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules More ❯

Are you passionate about Data and Analytics (D&A) and excited about how it can completely transform the way an enterprise works? Do you have the strategic vision, technical expertise, and leadership skills to drive data-driven solutions? Do you want to work in a dynamic, fast-growing category? If so, you might be the ideal candidate for … the role in the Data and Analytics function for Global Pet Nutrition (PN) at Mars. Pet Nutrition (PN) is the most vibrant category in the FMCG sector. As we work to transform this exciting category, a new program, Digital First, has been mobilized by the Mars Pet Nutrition (PN) leadership team. Digital First places pet parents at the centre … to demonstrate significant thought leadership, quality decision making, deep technical know-how, and an ability to navigate complex business challenges while building and leading a team of world class data and analytics leaders. With Digital First, PN is moving to a Product based model to create business facing digital capabilities. Develop and maintain robust data pipelines and storage More ❯

Senior Data Scientists/Analysts – SC/DV Cleared – (Military/Veteran-Friendly) — Multiple Openings 🌳 Location: London (Remote & Hybrid options) 🌳 Employment: Permanent 🌳 Clearance: Current SC or DV required 🌳 Make sense of mission-critical data that actually drives decisions. 🌳 Areti Group is partnering with one of the UK’s fastest-growing Series A-funded tech start-ups to hire … multiple Senior Data Scientists & Data Analysts . You’ll help deliver secure, high-impact analytics platforms for Defence, National Security, and Government projects. 🌳 Perfect for ex-Forces engineers (RAF, Army, Navy) or civilians who’ve worked in high-trust, high-stakes environments and want to use data science at scale . 🌳 What you’ll do 🌳 Build and … deploy machine learning models to solve real-world problems. Design data pipelines, analytics APIs, and decision dashboards used by senior stakeholders. Apply modern data science, AI, and big data techniques to defence and security missions. Deliver secure-by-design, production-grade analytics platforms for sensitive environments. Collaborate across engineering, security, and product teams to deliver at More ❯

Data Architect - Contract We are seeking a skilled Data Architect to lead the design, development, and oversight of our enterprise-wide data infrastructure. The Data Architect will involve data strategy and ensuring high standards of data quality, accessibility, and security across multiple systems and platforms. The ideal candidate will be a strategic thinker with … strong expertise in modern data architectures and data governance practices. Required Skills and Experience: Proven experience as a Data Architect or in a similar senior data-focused role. Extensive experience with various data modelling techniques and tools. Deep understanding of different data storage solutions (e.g., relational, NoSQL, data lakes) and their appropriate use … cases. Strong knowledge of data integration patterns, ETL/ELT processes, and data pipeline orchestration. Experience with AWS data platforms and their respective data services. Solid understanding of data governance principles, including data quality, metadata management, and access control. Familiarity with big data technologies (e.g., Spark, Hadoop) and distributed computing. Proficiency in More ❯

Databricks Data Engineer £500 - £600 per day Outside IR35 We're partnering with a leading online retail company that's transforming the way data and real-time intelligence shape customer experiences. Their mission is to harness cutting-edge data and streaming technologies to drive smarter decisions, improve efficiency, and create personalised journeys for millions of shoppers worldwide. … The Role As a Senior Data Engineer, you'll play a key role in developing and optimising the backbone of the company's data platform. You'll be responsible for building and maintaining large-scale, real-time data pipelines that power analytics, machine learning, and operational systems across the business. You'll collaborate with software engineers, data scientists, and analytics teams to ensure the platform delivers reliable, high-quality, and compliant data at scale. This is a hands-on engineering role that blends software craftsmanship with data architecture expertise. Key responsibilities: Design and implement high-throughput data streaming solutions using Kafka, Flink, or Confluent. Build and maintain scalable backend systems in Python or More ❯

Join us as a Data Platform Engineer for our client. At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector – we are always in search of the best talent … benefits, you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries. The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and on-premises … platforms. You will play a key role in ensuring that all data systems comply with industry regulations and security standards while enabling efficient access for analytics and operational teams. A strong command of Apache NiFi is essential for this role. You will be expected to design, implement, and maintain data flows using NiFi, ensuring accurate, efficient, and secure More ❯

Join us as a Data Platform Engineer for our client. At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector – we are always in search of the best talent … benefits, you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries. The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and on-premises … platforms. You will play a key role in ensuring that all data systems comply with industry regulations and security standards while enabling efficient access for analytics and operational teams. A strong command of Apache NiFi is essential for this role. You will be expected to design, implement, and maintain data flows using NiFi, ensuring accurate, efficient, and secure More ❯

Contract Opportunity: Data Engineer - 6-Month Inside IR35 Location: Milton Keynes (Hybrid - 3 Days in Office per Week) Duration: 6 Months IR35 Status: Inside IR35 Rate: Flexible, Depending on Experience We're looking for an experienced Data Engineer to join a project-focused team working on the development and implementation of a robust data model for banking … The engagement is inside IR35 , and rates are flexible based on experience. Role Purpose: You'll collaborate with a small technical team and liaise with IT to ensure the data model is effectively productionised. The ideal candidate will have strong technical data analysis skills, experience with big data technologies, and the ability to communicate complex insights … clearly to stakeholders. Key Responsibilities: Collaborate with a small team to build and refine a banking-focused data model. Liaise with IT teams to transition data models into production environments. Conduct data mining and exploratory data analysis to support model development. Apply strong SQL, Hadoop, and cloud-based data processing skills to manage and analyse More ❯

Responsibilities Designing, developing, and maintaining high-performance, scalable, and secure backend services and APIs using technologies such as Python and NodeJS. Collaborating with science teams, full stack engineers, and data engineers to understand data requirements and integrate user needs into backend services. Building and maintaining RESTful APIs to surface data in the platform, ensuring it is findable … and accessible in line with FAIR principles. Developing APIs that support future interoperability and federation for pathogen diagnostics, epidemiology monitoring, and research. Working with data engineers to build pipelines and processes that store and transform platform data for easy consumption. Implementing and managing data storage solutions including data lakes, data warehouses, and databases such as … object storage, Oracle Autonomous Database, and PostgreSQL. Supporting innovation efforts by exploring new technologies such as vector databases to enable search and AI use cases. Using big data technologies like Kafka, Iceberg, and Parquet, along with managed databases including PostgreSQL and Oracle vector databases. Operating, monitoring, and maintaining Oracle Cloud infrastructure to ensure backend services are highly available More ❯

Areti is currently seeking five Senior Data Scientists/Analysts to join one of the UK’s fastest-growing Series A-funded tech startups based in London . This is an exciting opportunity to be part of a dynamic, mission-driven company working on high-impact Defence and Government projects. Extensive training provided – including Palantir – making this a career … defining role . 🌳🌳🌳 The Role: You will work within a tight-knit, agile team tackling real-world problems using cutting-edge data science and engineering techniques. Key Responsibilities: Develop and deploy machine learning models Solve complex data problems in a big data environment Build scalable data pipelines and APIs Ideal Candidate Will Have: Previous experience … as a Data Scientist or Data Engineer Strong command of Python (including libraries such as scikit-learn, NumPy, matplotlib) Experience in deep learning frameworks such as TensorFlow or PyTorch Knowledge of SQL and relational databases; experience with Big Data environments Familiarity with API development and NoSQL databases Understanding of CI/CD practices , AWS , and GIT More ❯

Exp: 10 + yrs Job Summary The Senior Test Specialist/Architect in Cloud Automation Testing focuses on automating testing processes in ETL (Extract, Transform, Load) and Data Warehousing environments to ensure the quality and reliability of data pipelines. The role involves designing, developing, and implementing automation scripts for testing data transformations, data loading processes, and … data quality verification. Key Responsibilities 1. Develop and execute automated test scripts for etl processes and data pipelines. 2. Collaborate with cross functional teams to design and implement automated testing frameworks. 3. Create test plans and test cases for etl and data warehouse testing. 4. Identify and troubleshoot issues in data transformations and data loading … processes. 5. Conduct performance testing and ensure scalability of data pipelines. 6. Implement best practices for data quality assurance in etl environments. Skill Requirements 1. Proficiency in cloud-based automation testing tools like selenium, appium, or similar. 2. Experience in automation testing of etl (extract, transform, load) processes and data warehousing. 3. Strong understanding of sql for More ❯

GCP Data Engineer | London | £50,000 - £70,000 Peaple Talent have partnered with a specialist data consultancy delivering services across data engineering, data strategy, data migration, BI & analytics. With a diverse portfolio of clients and cutting-edge projects, our client is a trusted name in the data consulting space. Due to exciting growth plans … we are now looking for a Data Engineer , specialising in Google Cloud Platform (GCP) and BigQuery . We are looking for: Demonstrable data engineering/BI skills, with a focus on having delivered solutions in Google Cloud Platform (GCP) Strong experience designing and delivering data solutions using BigQuery Proficient in SQL and Python Experience working with Big Data technologies such as Apache Spark or PySpark Excellent communication skills, with the ability to engage effectively with senior stakeholders Nice to haves: GCP Data Engineering certifications BigQuery or other GCP tool certifications What’s in it for you: 📍 Location: London (Hybrid) 💻 Remote working: Occasional office visits each month 💰 Salary: £50,000–£70,000 DOE 🤝 Collaborative culture More ❯

We are seeking a Data Engineer (Databricks) to support the growth of a global technology provider within the Insurtech space. The role focuses on designing and delivering ETL pipelines and scalable solutions using the Azure ecosystem, with an emphasis on enabling advanced analytics and data-driven decision-making. As a key player in a high-performing data engineering team, you will contribute to large-scale transformation initiatives within the P&C Insurance space by developing robust data models, optimising data flows, and ensuring the accuracy and accessibility of critical information. The position requires close collaboration with both technical and business stakeholders, and is an excellent opportunity to join a team that invests in your … growth, with comprehensive training and certification programs and a real opportunity to showcase your talents. Key Responsibilities Design and develop ETL pipelines using Azure Data Factory for data ingestion and transformation Work with Azure Data Lake, Synapse, and SQL DW to manage large volumes of data Develop data transformation logic using SQL, Python, and PySpark More ❯

Databricks Data Engineer | London | £50,000 - £70,000 🧱 Peaple Talent have partnered with a specialist data consultancy delivering services across data engineering, data strategy, data migration, BI & analytics. As a trusted Microsoft Partner, my client are leaders in the industry, with a diverse portfolio of clients and projects. Due to exciting growth plans we are … now looking for a Data Engineer, specialising in Databricks. We are looking for: Demonstratable data engineering/BI skills, with a focus on having delivered in Microsoft Azure Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication … skills, effectively participating with Senior Stakeholders Nice to haves: Azure Data Engineering certifications Databricks certifications What's in it for you: 📍Location: London (Hybrid) 💻Remote working: Occasional office visits each month 💰Salary: £50,000-£70,000 DOE 🤝Collaborative culture and great team support 📈Vast L&D opportunities both internally and externally More ❯

Areti is seeking Senior Data Scientists/Data Engineers (Palantir, Python, Data Science) to work for a series A Funded tech start-up business based in the London area a very quick start. YOU MUST BE DV OR SC CLEARED You will be working on a number of large Defence and Government projects and be given the … opportunity to lead a number of high-profile programs. The Data Scientists will apply their technical skillsets and knowledge to solve exciting real-world problems in a small and dynamic company. You will work as part of a dynamic and close-knit team building new and proprietary approaches to solving multiple problems across the industry. The ideal candidates will … have a skillset to include the following: Experience in working as a Data Scientist/Data Engineer A understand of Palantir would be a huge + Experience of completing projects to deadline. Experience in Deep Learning and DL frameworks such as Tensorflow/Pytroch Deploying ML models Good command of Python and use of libraries for data More ❯

Energy Domain is a must. Under 7 years of experience only. Permanent. Key Responsibilities Design, develop, and maintain data ingestion pipelines using open-source frameworks and tools Build and optimise ETL/ELT processes to handle small to large-scale data processing requirements Develop data models and schemas that support analytics, business intelligence and product needs Monitor … troubleshoot, and optimise data pipeline performance and reliability Collaborate with stakeholders, analysts and product team to understand data requirements Implement data quality checks and validation processes to ensure data integrity Participate in architecture decisions and contribute to technical roadmap planning Technical Skills: Great SQL skills with experience in complex query optimization Strong Python programming skills with … experience in data processing libraries (pandas, NumPy, Apache Spark) Hands-on experience building and maintaining data ingestion pipelines Proven track record of optimising queries, code, and system performance Experience with open-source data processing frameworks (Apache Spark, Apache Kafka, Apache Airflow) Knowledge of distributed computing concepts and big data technologies Experience with version control systems More ❯

Data Analyst – £285/day | 10-Month Contract | Hybrid (London) We’re looking for an experienced Data Analyst to join a global Customer Success Data Science team , working on initiatives that enhance customer experience and operational excellence across international markets. This is an exciting opportunity to make a real impact by building data foundations, enabling actionable … insights, and driving automation across multiple regions. You’ll collaborate closely with cross-functional teams to ensure data drives strategy, informs decision-making, and helps deliver exceptional customer outcomes. What You’ll Do Build strong data foundations: Develop pipelines, improve data integrity, and establish clear metric definitions and governance across multiple international markets. Automate and streamline reporting … operational inefficiencies, turning insights into actionable recommendations. Support experimentation: Analyse A/B tests and digital experience initiatives to measure impact and inform continuous improvement. Tell the story with data: Communicate insights effectively to both technical and non-technical audiences, influencing strategy and driving better decisions. What We’re Looking For 8+ years of experience in Data Analytics More ❯

Our client is a leading hedge fund in London, leveraging cutting-edge technology and data to drive investment decisions across diverse asset classes. They're looking for a skilled Data Engineer to play a crucial role in building and optimizing the firm's data infrastructure. Responsibilities: Design, develop, and maintain robust data pipelines and ETL processes. … Collaborate with data scientists and analysts to support data-driven strategies. Ensure data accuracy, consistency, and security across various data sources. Drive innovation by exploring new technologies and best practices in data engineering. Requirements: Bachelor’s or Master’s in Computer Science, Engineering, or a related field. 1+ years of experience in data engineering. … Proficiency in SQL, Python, and big data tools like Spark. Excellent problem-solving skills and a collaborative mindset. More ❯

and these values are upheld only because of our people. Your role As a Confluent Consulting Engineer, you will be responsible for designing, developing, and maintaining scalable real-time data pipelines and integrations using Kafka and Confluent components. You will collaborate with data engineers, architects, and DevOps teams to deliver robust streaming solutions. Required: • Hands-on experience with … Apache Kafka (any distribution: open-source, Confluent, Cloudera, AWS MSK, etc.) • Strong proficiency in Java, Python, or Scala • Solid understanding of event-driven architecture and data streaming patterns • Experience deploying Kafka on cloud platforms such as AWS, GCP, or Azure • Familiarity with Docker, Kubernetes, and CI/CD pipelines • Excellent problem-solving and communication abilities Preferred: Candidates with experience … with Kafka Connect, Kafka Streams, KSQL, Schema Registry, REST Proxy, Confluent Control Center • Hands-on with Confluent Cloud services, including ksqlDB Cloud and Apache Flink • Familiarity with Stream Governance, Data Lineage, Stream Catalog, Audit Logs, RBAC • Confluent certifications (Developer, Administrator, or Flink Developer) • Experience with Confluent Platform, Confluent Cloud managed services, multi-cloud deployments, and Confluent for Kubernetes • Knowledge More ❯

About Us: Quantiphi is an award-winning Applied AI and Big Data software and services company, driven by a deep desire to solve transformational problems at the heart of businesses. Our signature approach combines groundbreaking machine-learning research with disciplined cloud and data-engineering practices to create breakthrough impact at unprecedented speed. Company Highlights: Quantiphi, a Services … AI Platforms company, has seen 2.5x growth YoY since its inception in 2013 Headquartered in Boston, with 3300+ data science professionals across 11 global offices Winner of 3X NVIDIA AI Partner of the year award Winner of the 17X Google Cloud Partner of the Year award including "Machine Learning, Breakthrough and Social Impact partner Winner of 3X AWS AI … Recognized Leaders by Gartner, Forrester, IDC, ISG, Everest Group and other leading analyst and independent research firms. Recognized in the Fast Company 2021 World Changing Ideas- AI and Data category Winner of 2021 AWS Canada Rising Star of the Year Be part of a trailblazing team that’s shaping the future of AI, ML, and cloud innovation. Your next More ❯

About Tredence: Tredence is a global analytics services and solutions company. Our capabilities range from Data Visualization, Data Management to Advanced analytics, Big Data and Machine Learning. Our uniqueness is in bringing the right mix of technology and business analytics to create sustainable white-box solutions that are transitioned to our clients at the end of … the engagement. We do this cost effectively using a global execution model leveraging our clients' existing technology and data assets. We also come in with strong IP and pre-built analytics solutions in data mining, business intelligence and Big Data. Role Overview As a Client Partner, you’ll work hand-in-hand with senior leaders to nurture … and create lasting impact by helping our clients achieve their business goals You will act as the bridge between business, delivery, and solution teams — ensuring that advanced AI and data-driven solutions translate into measurable business impact for global retail clients. Key Responsibilities Client Relationship Management Serve as the primary point of contact for day-to-day client interactions. More ❯

Position: Senior AI/ML and Data Science Developer Employment Type: Permanent Start: ASAP (October/November/Decmeber 2025) Location: Hertfordshire, UK - Hybrid Language(s): English Salary: Up to £80,000 p/a + bonus and benefits Focus on SAP is a specialist SAP and ERP Recruitment organisation offering both permanent and contract staffing solutions on a … communities they serve. For you, this means working on challenging projects that demand innovation, collaboration, and thought leadership . Role – We are seeking an experienced AI/ML and Data Science Developer to design, develop, and deploy AI/ML models and solutions, including LLMs and Generative AI applications. You will work across the full development lifecycle, from concept … and feature engineering to deployment and monitoring by driving innovation and helping shape next-generation data-driven solutions. You will collaborate with business and technical stakeholders to translate requirements into scalable, production-ready AI/ML systems, using advanced data science techniques to deliver measurable business impact. Key Responsibilities: Design, develop, and deploy AI/ML and GenAI More ❯

Position: Senior AI/ML and Data Science Developer Employment Type: Permanent Start: ASAP (October/November/Decmeber 2025) Location: Hertfordshire, UK - Hybrid Language(s): English Salary: Up to £80,000 p/a + bonus and benefits Focus on SAP is a specialist SAP and ERP Recruitment organisation offering both permanent and contract staffing solutions on a … communities they serve. For you, this means working on challenging projects that demand innovation, collaboration, and thought leadership . Role – We are seeking an experienced AI/ML and Data Science Developer to design, develop, and deploy AI/ML models and solutions, including LLMs and Generative AI applications. You will work across the full development lifecycle, from concept … and feature engineering to deployment and monitoring by driving innovation and helping shape next-generation data-driven solutions. You will collaborate with business and technical stakeholders to translate requirements into scalable, production-ready AI/ML systems, using advanced data science techniques to deliver measurable business impact. Key Responsibilities: Design, develop, and deploy AI/ML and GenAI More ❯

We're the world's leading data, insights, and consulting company; we shape the brands of tomorrow by better understanding people everywhere. Kantar's Profiles division is home to the world's largest audience network. With access to 170m+ people in over 100 global markets, we offer unrivalled global reach with local relevancy. Validated by industry leading anti-fraud … technology, Kantar's Profiles Audience Network delivers the most meaningful data with consistency, accuracy, and accountability – all at speed and scale. Job Details Location: Reading or London, UK Please note: You must have the right to work in the UK. We are unable to offer visa sponsorship for this role. At Kantar, we help the world's leading brands … make better decisions through data. We're now looking for an experienced Lead Data Engineer to join our team and help us elevate the quality and integrity of our global consumer panels. These panels—more than 50 worldwide—are the foundation of our data-driven insights. Your mission? To build and evolve the data ecosystem, pipelines and More ❯

Data Engineer (Insurance OR Financial Services Background) - London - £75,000-£85,000 - Hybrid Data Engineer required to join a UK leading insurance business based in London. They are looking for someone who is happy to collaborate on a hybrid basis (1/2 days a week onsite). They offer excelllent career advancement alongside regular salary reviews. Role … Overview This position is perfect for someone who takes pride in owning data, code, and outcomes. You’ll play a central role in shaping and maintaining the data platforms and pipelines that drive pricing analytics and underwriting performance. Your work will ensure data quality, consistency, and scalability across the board. Partnering with teams across Pricing and Underwriting … Data Science, and Engineering, you’ll design and build modular, transparent pipelines - delivering end-to-end visibility from raw data ingestion to pricing model deployment, empowering smarter and faster decision-making. Key Responsibilities Take ownership of, enhance, and scale the data platforms and pipelines that underpin pricing analytics. Implement robust governance, validation, and alerting systems to maintain More ❯