1 to 25 of 98 Big Data Jobs in the South East

Job Title: Senior Data Engineer Location: London, UK (3 days in the office) SC Cleared: Required Job Type: Full-Time Experience: 8+ years Job Summary : We are seeking a highly skilled and experienced Senior Data Engineer to join our team and contribute to the development and maintenance of our cutting-edge Azure Databricks platform for economic data. This … platform is critical for our Monetary Analysis, Forecasting, and Modelling activities. The Senior Data Engineer will be responsible for building and optimising data pipelines, implementing data transformations, and ensuring data quality and reliability. This role requires a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working … with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation: Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data providers) into the Azure Databricks platform. Optimise data pipelines for performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules More ❯

Job Title: Senior Data Engineer Location: London, UK (3 days in the office) SC Cleared: Required Job Type: Full-Time Experience: 8+ years Job Summary : We are seeking a highly skilled and experienced Senior Data Engineer to join our team and contribute to the development and maintenance of our cutting-edge Azure Databricks platform for economic data. This … platform is critical for our Monetary Analysis, Forecasting, and Modelling activities. The Senior Data Engineer will be responsible for building and optimising data pipelines, implementing data transformations, and ensuring data quality and reliability. This role requires a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working … with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation: Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data providers) into the Azure Databricks platform. Optimise data pipelines for performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules More ❯

Job Title: Senior Data Engineer Location: London, UK (3 days in the office) SC Cleared: Required Job Type: Full-Time Experience: 8+ years Job Summary : We are seeking a highly skilled and experienced Senior Data Engineer to join our team and contribute to the development and maintenance of our cutting-edge Azure Databricks platform for economic data. This … platform is critical for our Monetary Analysis, Forecasting, and Modelling activities. The Senior Data Engineer will be responsible for building and optimising data pipelines, implementing data transformations, and ensuring data quality and reliability. This role requires a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working … with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation: Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data providers) into the Azure Databricks platform. Optimise data pipelines for performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules More ❯

Role Title: Big Data with Java Location: London, UK(Hybrid) Employment Type: Contract Job Description: Java - Must have Big Data – must have Interview includes coding test. Job Description: Scala/Spark • Good Big Data resource with the below Skillset: Java Big data technologies. • Linux Based Hadoop Ecosystem (HDFS, Impala, Hive, HBase, etc. … Experience in Big data technologies, real time data processing platform (Spark Streaming) experience would be an advantage. • Consistently demonstrates clear and concise written and verbal communication • A history of delivering against agreed objectives • Ability to multi-task and work under pressure • Demonstrated problem solving and decision-making skills • Excellent analytical and process-based skills, i.e. process flow More ❯

relevant skills and experience, we're looking for people who are innovative, commercial and value the work that they do. Our London office is currently looking for a Senior Data Engineer. The Role The Senior Data Engineer will be responsible for designing, developing, and maintaining the infrastructure and systems required for data storage, processing, and analysis. They … play a crucial role in building and managing the data pipelines that enable efficient and reliable data integration, transformation, and delivery for all data users across the enterprise. Key Responsibilities Designs and develops data pipelines that extract data from various sources, transform it into the desired format, and load it into the appropriate data storage systems Collaborates with data scientists and analysts to optimize models and algorithms for data quality, security, and governance Integrates data from different sources, including databases, data warehouses, APIs, and external systems Ensures data consistency and integrity during the integration process, performing data validation and cleaning as needed Transforms raw data into More ❯

Position: Analytics Engineer Location: London (hybrid 3 times a week)Department: ITType: 3 months rolling contractOutside IR35 The client is seeking an experienced Analytics Engineer to join its Data & Analytics team. The role focuses on building scalable data pipelines, transforming raw data into clean, reusable datasets, and enabling data-driven decision-making. Key Responsibilities Design and … build data products, with proficiency throughout the data lifecycle. Develop robust data models through close collaboration with business users and the engineering team. Partner with senior management, operational leads, and other stakeholders, coaching and supporting a data-driven culture, including KPI definition and reporting frameworks. Accountable for data extraction, transforming JSON & XML, utlising high experience … within metadata management. Collaborate with data engineers to develop data, enrich product design and integrate data for predictive models or machine learning. Deliver well-defined, transformed, tested, documented, and code-reviewed datasets for analysis. Evaluate and recommend improvements in data flow, influencing and supporting architects and engineers. To work independently and to manage multiple data More ❯

AWS Data Architect - London/Remote - £500/pd Please note - this role will require you to be UK based with the unrestricted right to work in the UK. You must be willing to work on-site in London as required. About the Role We are seeking an experienced Data Architect to design, develop, and govern our enterprise … data landscape. This is a critical role in shaping our data strategy, ensuring data quality, accessibility, and security across multiple platforms and systems. If you thrive on building robust data ecosystems and delivering impactful solutions, we want to hear from you. Key Responsibilities Design and implement scalable, secure, and efficient data architectures. Define and enforce … data governance standards, including data quality and metadata management. Collaborate with stakeholders to align data solutions with business objectives. Oversee data integration, ETL/ELT processes, and pipeline orchestration. Optimize data storage solutions across relational, NoSQL, and data lake environments. Leverage AWS data platforms and services to deliver enterprise-grade solutions. Stay ahead More ❯

Are you passionate about Data and Analytics (D&A) and excited about how it can completely transform the way an enterprise works? Do you have the strategic vision, technical expertise, and leadership skills to drive data-driven solutions? Do you want to work in a dynamic, fast-growing category? If so, you might be the ideal candidate for … the role in the Data and Analytics function for Global Pet Nutrition (PN) at Mars. Pet Nutrition (PN) is the most vibrant category in the FMCG sector. As we work to transform this exciting category, a new program, Digital First, has been mobilized by the Mars Pet Nutrition (PN) leadership team. Digital First places pet parents at the centre … to demonstrate significant thought leadership, quality decision making, deep technical know-how, and an ability to navigate complex business challenges while building and leading a team of world class data and analytics leaders. With Digital First, PN is moving to a Product based model to create business facing digital capabilities. Develop and maintain robust data pipelines and storage More ❯

Are you passionate about Data and Analytics (D&A) and excited about how it can completely transform the way an enterprise works? Do you have the strategic vision, technical expertise, and leadership skills to drive data-driven solutions? Do you want to work in a dynamic, fast-growing category? If so, you might be the ideal candidate for … the role in the Data and Analytics function for Global Pet Nutrition (PN) at Mars. Pet Nutrition (PN) is the most vibrant category in the FMCG sector. As we work to transform this exciting category, a new program, Digital First, has been mobilized by the Mars Pet Nutrition (PN) leadership team. Digital First places pet parents at the centre … to demonstrate significant thought leadership, quality decision making, deep technical know-how, and an ability to navigate complex business challenges while building and leading a team of world class data and analytics leaders. With Digital First, PN is moving to a Product based model to create business facing digital capabilities. Develop and maintain robust data pipelines and storage More ❯

Are you passionate about Data and Analytics (D&A) and excited about how it can completely transform the way an enterprise works? Do you have the strategic vision, technical expertise, and leadership skills to drive data-driven solutions? Do you want to work in a dynamic, fast-growing category? If so, you might be the ideal candidate for … the role in the Data and Analytics function for Global Pet Nutrition (PN) at Mars. Pet Nutrition (PN) is the most vibrant category in the FMCG sector. As we work to transform this exciting category, a new program, Digital First, has been mobilized by the Mars Pet Nutrition (PN) leadership team. Digital First places pet parents at the centre … to demonstrate significant thought leadership, quality decision making, deep technical know-how, and an ability to navigate complex business challenges while building and leading a team of world class data and analytics leaders. With Digital First, PN is moving to a Product based model to create business facing digital capabilities. Develop and maintain robust data pipelines and storage More ❯

Databricks Data Engineer £500 - £600 per day Outside IR35 We're partnering with a leading online retail company that's transforming the way data and real-time intelligence shape customer experiences. Their mission is to harness cutting-edge data and streaming technologies to drive smarter decisions, improve efficiency, and create personalised journeys for millions of shoppers worldwide. … The Role As a Senior Data Engineer, you'll play a key role in developing and optimising the backbone of the company's data platform. You'll be responsible for building and maintaining large-scale, real-time data pipelines that power analytics, machine learning, and operational systems across the business. You'll collaborate with software engineers, data scientists, and analytics teams to ensure the platform delivers reliable, high-quality, and compliant data at scale. This is a hands-on engineering role that blends software craftsmanship with data architecture expertise. Key responsibilities: Design and implement high-throughput data streaming solutions using Kafka, Flink, or Confluent. Build and maintain scalable backend systems in Python or More ❯

Join us as a Data Platform Engineer for our client. At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector – we are always in search of the best talent … benefits, you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries. The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and on-premises … platforms. You will play a key role in ensuring that all data systems comply with industry regulations and security standards while enabling efficient access for analytics and operational teams. A strong command of Apache NiFi is essential for this role. You will be expected to design, implement, and maintain data flows using NiFi, ensuring accurate, efficient, and secure More ❯

Join us as a Data Platform Engineer for our client. At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector – we are always in search of the best talent … benefits, you will be deployed across our portfolio of clients as a specialist consultant, working on a wide array of complex projects across multiple industries. The Role: As a Data Platform Engineer in a highly regulated environment, you will be responsible for designing, building, and maintaining secure and scalable data infrastructure that supports both cloud and on-premises … platforms. You will play a key role in ensuring that all data systems comply with industry regulations and security standards while enabling efficient access for analytics and operational teams. A strong command of Apache NiFi is essential for this role. You will be expected to design, implement, and maintain data flows using NiFi, ensuring accurate, efficient, and secure More ❯

Data Architect - Contract We are seeking a skilled Data Architect to lead the design, development, and oversight of our enterprise-wide data infrastructure. This role plays a key part in defining our data strategy and ensuring high standards of data quality, accessibility, and security across multiple systems and platforms. The ideal candidate will be a … strategic thinker with strong expertise in modern data architectures and data governance practices. Required Skills and Experience: Proven experience as a Data Architect or in a similar senior data-focused role. Extensive experience with various data modelling techniques and tools. Deep understanding of different data storage solutions (e.g., relational, NoSQL, data lakes) and … their appropriate use cases. Strong knowledge of data integration patterns, ETL/ELT processes, and data pipeline orchestration. Experience with AWS data platforms and their respective data services. Solid understanding of data governance principles, including data quality, metadata management, and access control. Familiarity with big data technologies (e.g., Spark, Hadoop) and distributed More ❯

Contract Opportunity: Data Engineer - 6-Month Inside IR35 Location: Milton Keynes (Hybrid - 3 Days in Office per Week) Duration: 6 Months IR35 Status: Inside IR35 Rate: Flexible, Depending on Experience We're looking for an experienced Data Engineer to join a project-focused team working on the development and implementation of a robust data model for banking … The engagement is inside IR35 , and rates are flexible based on experience. Role Purpose: You'll collaborate with a small technical team and liaise with IT to ensure the data model is effectively productionised. The ideal candidate will have strong technical data analysis skills, experience with big data technologies, and the ability to communicate complex insights … clearly to stakeholders. Key Responsibilities: Collaborate with a small team to build and refine a banking-focused data model. Liaise with IT teams to transition data models into production environments. Conduct data mining and exploratory data analysis to support model development. Apply strong SQL, Hadoop, and cloud-based data processing skills to manage and analyse More ❯

Responsibilities Designing, developing, and maintaining high-performance, scalable, and secure backend services and APIs using technologies such as Python and NodeJS. Collaborating with science teams, full stack engineers, and data engineers to understand data requirements and integrate user needs into backend services. Building and maintaining RESTful APIs to surface data in the platform, ensuring it is findable … and accessible in line with FAIR principles. Developing APIs that support future interoperability and federation for pathogen diagnostics, epidemiology monitoring, and research. Working with data engineers to build pipelines and processes that store and transform platform data for easy consumption. Implementing and managing data storage solutions including data lakes, data warehouses, and databases such as … object storage, Oracle Autonomous Database, and PostgreSQL. Supporting innovation efforts by exploring new technologies such as vector databases to enable search and AI use cases. Using big data technologies like Kafka, Iceberg, and Parquet, along with managed databases including PostgreSQL and Oracle vector databases. Operating, monitoring, and maintaining Oracle Cloud infrastructure to ensure backend services are highly available More ❯

The Role The Data Practice is the team made up of experts dedicated to Data centric offers (Data platform, Data Analytics, IoT, Big Data, Data compliance) who support our customers in their IT and Data transformations. We help customers choose the right tools and services, build highly scalable secure environments, and help More ❯

Data QA Engineer – Remote-first - £55-65,000 Overview: As a Data QA Engineer, you will ensure the reliability, accuracy and performance of our client’s data solutions. Operating remotely, you will work closely with Data Engineers, Architects and Analysts to design and execute automated testing strategies that validate data pipelines, integrations and transformations across … cloud platforms. Your work will underpin data quality and ensure that insights are built on trusted, consistent data. Key Responsibilities: Design and implement automated tests for data pipelines, ETL processes and APIs across AWS and Azure environments. Develop and maintain test frameworks that validate data quality, completeness and integrity. Collaborate with engineering teams to identify data defects, root causes and performance issues. Support continuous integration and deployment by embedding automated testing into CI/CD pipelines. Validate source-to-target data mapping, transformations and business logic. Produce clear test documentation and communicate test results to technical and non-technical stakeholders. Monitor data test coverage and define quality metrics for ongoing improvement. Contribute to More ❯

Exp: 10 + yrs Job Summary The Senior Test Specialist/Architect in Cloud Automation Testing focuses on automating testing processes in ETL (Extract, Transform, Load) and Data Warehousing environments to ensure the quality and reliability of data pipelines. The role involves designing, developing, and implementing automation scripts for testing data transformations, data loading processes, and … data quality verification. Key Responsibilities 1. Develop and execute automated test scripts for etl processes and data pipelines. 2. Collaborate with cross functional teams to design and implement automated testing frameworks. 3. Create test plans and test cases for etl and data warehouse testing. 4. Identify and troubleshoot issues in data transformations and data loading … processes. 5. Conduct performance testing and ensure scalability of data pipelines. 6. Implement best practices for data quality assurance in etl environments. Skill Requirements 1. Proficiency in cloud-based automation testing tools like selenium, appium, or similar. 2. Experience in automation testing of etl (extract, transform, load) processes and data warehousing. 3. Strong understanding of sql for More ❯

Exp: 10 + yrs Job Summary The Senior Test Specialist/Architect in Cloud Automation Testing focuses on automating testing processes in ETL (Extract, Transform, Load) and Data Warehousing environments to ensure the quality and reliability of data pipelines. The role involves designing, developing, and implementing automation scripts for testing data transformations, data loading processes, and … data quality verification. Key Responsibilities 1. Develop and execute automated test scripts for etl processes and data pipelines. 2. Collaborate with cross functional teams to design and implement automated testing frameworks. 3. Create test plans and test cases for etl and data warehouse testing. 4. Identify and troubleshoot issues in data transformations and data loading … processes. 5. Conduct performance testing and ensure scalability of data pipelines. 6. Implement best practices for data quality assurance in etl environments. Skill Requirements 1. Proficiency in cloud-based automation testing tools like selenium, appium, or similar. 2. Experience in automation testing of etl (extract, transform, load) processes and data warehousing. 3. Strong understanding of sql for More ❯

Exp: 10 + yrs Job Summary The Senior Test Specialist/Architect in Cloud Automation Testing focuses on automating testing processes in ETL (Extract, Transform, Load) and Data Warehousing environments to ensure the quality and reliability of data pipelines. The role involves designing, developing, and implementing automation scripts for testing data transformations, data loading processes, and … data quality verification. Key Responsibilities 1. Develop and execute automated test scripts for etl processes and data pipelines. 2. Collaborate with cross functional teams to design and implement automated testing frameworks. 3. Create test plans and test cases for etl and data warehouse testing. 4. Identify and troubleshoot issues in data transformations and data loading … processes. 5. Conduct performance testing and ensure scalability of data pipelines. 6. Implement best practices for data quality assurance in etl environments. Skill Requirements 1. Proficiency in cloud-based automation testing tools like selenium, appium, or similar. 2. Experience in automation testing of etl (extract, transform, load) processes and data warehousing. 3. Strong understanding of sql for More ❯

GCP Data Engineer | London | £50,000 - £70,000 Peaple Talent have partnered with a specialist data consultancy delivering services across data engineering, data strategy, data migration, BI & analytics. With a diverse portfolio of clients and cutting-edge projects, our client is a trusted name in the data consulting space. Due to exciting growth plans … we are now looking for a Data Engineer , specialising in Google Cloud Platform (GCP) and BigQuery . We are looking for: Demonstrable data engineering/BI skills, with a focus on having delivered solutions in Google Cloud Platform (GCP) Strong experience designing and delivering data solutions using BigQuery Proficient in SQL and Python Experience working with Big Data technologies such as Apache Spark or PySpark Excellent communication skills, with the ability to engage effectively with senior stakeholders Nice to haves: GCP Data Engineering certifications BigQuery or other GCP tool certifications What’s in it for you: 📍 Location: London (Hybrid) 💻 Remote working: Occasional office visits each month 💰 Salary: £50,000–£70,000 DOE 🤝 Collaborative culture More ❯

GCP Data Engineer | London | £50,000 - £70,000 Peaple Talent have partnered with a specialist data consultancy delivering services across data engineering, data strategy, data migration, BI & analytics. With a diverse portfolio of clients and cutting-edge projects, our client is a trusted name in the data consulting space. Due to exciting growth plans … we are now looking for a Data Engineer , specialising in Google Cloud Platform (GCP) and BigQuery . We are looking for: Demonstrable data engineering/BI skills, with a focus on having delivered solutions in Google Cloud Platform (GCP) Strong experience designing and delivering data solutions using BigQuery Proficient in SQL and Python Experience working with Big Data technologies such as Apache Spark or PySpark Excellent communication skills, with the ability to engage effectively with senior stakeholders Nice to haves: GCP Data Engineering certifications BigQuery or other GCP tool certifications What’s in it for you: 📍 Location: London (Hybrid) 💻 Remote working: Occasional office visits each month 💰 Salary: £50,000–£70,000 DOE 🤝 Collaborative culture More ❯

GCP Data Engineer | London | £50,000 - £70,000 Peaple Talent have partnered with a specialist data consultancy delivering services across data engineering, data strategy, data migration, BI & analytics. With a diverse portfolio of clients and cutting-edge projects, our client is a trusted name in the data consulting space. Due to exciting growth plans … we are now looking for a Data Engineer , specialising in Google Cloud Platform (GCP) and BigQuery . We are looking for: Demonstrable data engineering/BI skills, with a focus on having delivered solutions in Google Cloud Platform (GCP) Strong experience designing and delivering data solutions using BigQuery Proficient in SQL and Python Experience working with Big Data technologies such as Apache Spark or PySpark Excellent communication skills, with the ability to engage effectively with senior stakeholders Nice to haves: GCP Data Engineering certifications BigQuery or other GCP tool certifications What’s in it for you: 📍 Location: London (Hybrid) 💻 Remote working: Occasional office visits each month 💰 Salary: £50,000–£70,000 DOE 🤝 Collaborative culture More ❯

Databricks Data Engineer | London | £50,000 - £70,000 🧱 Peaple Talent have partnered with a specialist data consultancy delivering services across data engineering, data strategy, data migration, BI & analytics. As a trusted Microsoft Partner, my client are leaders in the industry, with a diverse portfolio of clients and projects. Due to exciting growth plans we are … now looking for a Data Engineer, specialising in Databricks. We are looking for: Demonstratable data engineering/BI skills, with a focus on having delivered in Microsoft Azure Strong experience designing and delivering data solutions in Databricks Proficient with SQL and Python Experience using Big Data technologies such as Apache Spark or PySpark Great communication … skills, effectively participating with Senior Stakeholders Nice to haves: Azure Data Engineering certifications Databricks certifications What's in it for you: 📍Location: London (Hybrid) 💻Remote working: Occasional office visits each month 💰Salary: £50,000-£70,000 DOE 🤝Collaborative culture and great team support 📈Vast L&D opportunities both internally and externally More ❯