1 to 25 of 125 Big Data Jobs in London

BIG DATA ENGINEER - DV CLEARED NEW PERMANENT JOB OPPORTUNITY AVAILABLE WITHIN A LEADING NATIONAL SECURITY SME FOR A BIG DATA ENGINEER WITH DV CLEARANCE Permanent job opportunity for a Big Data Engineer Leading National Security & Defence SME Salary up to £80,000 plus clearance bonus London based organisation in an easily accessible location To … apply please call or email WHO WE ARE? We are recruiting multiple Big Data Engineers to support urgent National Security & Defence projects in London. Due to the nature of these projects you must hold DV or enhanced DV Security Clearance. WHAT WILL THE BIG DATA ENGINEER BE DOING? You will be joining a leading SME who … is working hard to support National Security projects within UK Govt. Departments in London. As part of a team, you will be responsible for implementing Big Data Solutions in Mission-Critical areas. WE NEED THE BIG DATA ENGINEER TO HAVE.... Current DV clearance - Standard or Enhanced Must have experience with big data tools such More ❯

Junior Big Data Developer (Python & SQL Focus) 📊 !!! URGENT HIRING !!! Looking for London based Citizens/Dependents ***We DO NOT provide SPONSORSHIP*** We're looking for an enthusiastic and detail-oriented Junior Big Data Developer to join our data engineering team. This role is ideal for an early-career professional with foundational knowledge in data processing, strong proficiency in Python , and expert skills in SQL . You'll focus on building, testing, and maintaining data pipelines and ensuring data quality across our scalable Big Data platforms. Key Responsibilities Data Pipeline Development: Assist in the design, construction, and maintenance of robust ETL/ELT pipelines to integrate data from … various sources into our data warehouse or data lake. Data Transformation with Python: Write, optimize, and maintain production-grade Python scripts to clean, transform, aggregate, and process large volumes of data. Database Interaction (SQL): Develop complex, high-performance SQL queries (DDL/DML) for data extraction, manipulation, and validation within relational and data warehousing environments. More ❯

Junior Big Data Developer (Python & SQL Focus) 📊 !!! URGENT HIRING !!! Looking for London based Citizens/Dependents ***We DO NOT provide SPONSORSHIP*** We're looking for an enthusiastic and detail-oriented Junior Big Data Developer to join our data engineering team. This role is ideal for an early-career professional with foundational knowledge in data processing, strong proficiency in Python , and expert skills in SQL . You'll focus on building, testing, and maintaining data pipelines and ensuring data quality across our scalable Big Data platforms. Key Responsibilities Data Pipeline Development: Assist in the design, construction, and maintenance of robust ETL/ELT pipelines to integrate data from … various sources into our data warehouse or data lake. Data Transformation with Python: Write, optimize, and maintain production-grade Python scripts to clean, transform, aggregate, and process large volumes of data. Database Interaction (SQL): Develop complex, high-performance SQL queries (DDL/DML) for data extraction, manipulation, and validation within relational and data warehousing environments. More ❯

Job Title: Senior Data Engineer Location: London, UK (3 days in the office) SC Cleared: Required Job Type: Full-Time Experience: 8+ years Job Summary : We are seeking a highly skilled and experienced Senior Data Engineer to join our team and contribute to the development and maintenance of our cutting-edge Azure Databricks platform for economic data. This … platform is critical for our Monetary Analysis, Forecasting, and Modelling activities. The Senior Data Engineer will be responsible for building and optimising data pipelines, implementing data transformations, and ensuring data quality and reliability. This role requires a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working … with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data providers) into the Azure Databricks platform. Optimise data pipelines for performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules More ❯

Job Title: Senior Data Engineer Location: London, UK (3 days in the office) SC Cleared: Required Job Type: Full-Time Experience: 8+ years Job Summary : We are seeking a highly skilled and experienced Senior Data Engineer to join our team and contribute to the development and maintenance of our cutting-edge Azure Databricks platform for economic data. This … platform is critical for our Monetary Analysis, Forecasting, and Modelling activities. The Senior Data Engineer will be responsible for building and optimising data pipelines, implementing data transformations, and ensuring data quality and reliability. This role requires a strong understanding of data engineering principles, big data technologies, cloud computing (specifically Azure), and experience working … with large datasets. Key Responsibilities : Data Pipeline Development & Optimisation : Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data from various sources (e.g., APIs, databases, financial data providers) into the Azure Databricks platform. Optimise data pipelines for performance, efficiency, and cost-effectiveness. Implement data quality checks and validation rules More ❯

CONTRACT DATA ENGINEER - eDV CLEARED NEW OUTSIDE IR35 CONTRACT OPPORTUNITY AVAILABLE WITHIN A LEADING NATIONAL SECURITY SME FOR A DATA ENGINEER WITH eDV CLEARANCE Contract job opportunity for a Data Engineer National Security client Palantir Foundry Outside IR35 Central London based organisation in an easily accessible location To apply please call/or email WHO WE ARE … We are recruiting a contract Data Engineer to work with a National Security SME in central London. Due to the nature of the work, you must hold enhanced DV Clearance. WE NEED THE DATA ENGINEER TO HAVE.... Current enhanced DV Security Clearance Experience with big data tools such as Hadoop, Cloudera or Elasticsearch Experience With Palantir … Scrum environment with tools such as Confluence/Jira Experience in design, development, test and integration of software Willingness to learn new technologies IT WOULD BE NICE FOR THE DATA ENGINEER TO HAVE.... Cloud based architectures Microservice architecture or server-less architecture Messaging/routing technologies such as Apache Nifi/RabbitMQ TO BE CONSIDERED.... Please either apply by More ❯

DATA ARCHITECT - DV CLEARED NEW PERMANENT JOB OPPORTUNITY AVAILABLE WITHIN A LEADING NATIONAL SECURITY SME FOR A DATA ARCHITECT WITH ENHANCED DV CLEARANCE Permanent job opportunity for a Data Architect Leading National Security SME Salary up to £100,000 + Bonus London based organisation in an easily accessible location To apply please call or email WHO WE … ARE? We are recruiting multiple a Data Architect to support urgent National Security & Defence projects in London. Due to the nature of these projects you must hold enhanced DV Security Clearance. WHAT WILL THE DATA ARCHITECT BE DOING? You will be joining a leading SME who is working hard to support National Security projects within UK Govt. Departments … in London. As part of a team, you will be responsible for designing and implementing Data Solutions in Mission-Critical areas. WE NEED THE DATA ARCHITECT TO HAVE.... Current DV clearance - Enhanced Good at understanding complexity and abstracting that into a form that is consumable for a non-technical audience. Experience of achieving data interoperability between More ❯

The Global Director of Data and Information will be responsible for developing and executing a comprehensive data and information strategy that aligns with the organisation’s business goals. You will; help the organisation gain the benefit from the data within our business systems. identify and exploit opportunities to integrate operational and telemetry data into our Business … Intelligence ensure the organisation adheres to the key principles of information management, particularly Accountability, Integrity, Compliance and Usability. This role involves leading data governance, analytics, and data management initiatives, ensuring that data is effectively utilized to drive strategic decision-making across the organisation. The ideal candidate will possess strong leadership skills, a deep understanding of data architecture, and a proven ability to leverage data for business insights. Key Responsibilities: Strategy Development : Define and implement a global andData and Information strategy that aligns with the organisation’s overall objectives and priorities, considering and anticipating future scalability and compliance requirements. Team Management : Recruit, mentor, and develop a high-performing data team, promoting a culture of More ❯

The Global Director of Data and Information will be responsible for developing and executing a comprehensive data and information strategy that aligns with the organisation’s business goals. You will; help the organisation gain the benefit from the data within our business systems. identify and exploit opportunities to integrate operational and telemetry data into our Business … Intelligence ensure the organisation adheres to the key principles of information management, particularly Accountability, Integrity, Compliance and Usability. This role involves leading data governance, analytics, and data management initiatives, ensuring that data is effectively utilized to drive strategic decision-making across the organisation. The ideal candidate will possess strong leadership skills, a deep understanding of data architecture, and a proven ability to leverage data for business insights. Key Responsibilities: Strategy Development : Define and implement a global andData and Information strategy that aligns with the organisation’s overall objectives and priorities, considering and anticipating future scalability and compliance requirements. Team Management : Recruit, mentor, and develop a high-performing data team, promoting a culture of More ❯

For this role, you will be responsible for providing the framework that appropriately replicates the Big Data needs of a company utilizing data. Essential requirements: More than 3 years of presales experience in the design of Big Data and Data analytics solutions according to customer requirements Previous experience with the preparation of high-quality engaging … the offer solution team, solution definition, effort and cost estimation, Past experience in dealing with partners, tools vendors, etc. Business Domain Knowledge More than 5 years of experience in Big Data implementation projects Experience in the definition of Big Data architecture with different tools and environments: Cloud (AWS, Azure and GCP), Cloudera, No-sql databases (Cassandra … Mongo DB), ELK, Kafka, Snowflake, etc. Past experience in Data Engineering and data quality tools (Informatica, Talend, etc.) Previous involvement in working in a Multilanguage and multicultural environment Proactive, tech passionate and highly motivated Desirable requirements: Experience in Data analysis and visualization solutions: Microstrategy, Qlik, PowerBI, Tableau, Looker,... Background in Data Governance and Data More ❯

Position: Analytics Engineer Location: London (hybrid 3 times a week)Department: ITType: 3 months rolling contractOutside IR35 The client is seeking an experienced Analytics Engineer to join its Data & Analytics team. The role focuses on building scalable data pipelines, transforming raw data into clean, reusable datasets, and enabling data-driven decision-making. Key Responsibilities Design and … build data products, with proficiency throughout the data lifecycle. Develop robust data models through close collaboration with business users and the engineering team. Partner with senior management, operational leads, and other stakeholders, coaching and supporting a data-driven culture, including KPI definition and reporting frameworks. Accountable for data extraction, transforming JSON & XML, utlising high experience … within metadata management. Collaborate with data engineers to develop data, enrich product design and integrate data for predictive models or machine learning. Deliver well-defined, transformed, tested, documented, and code-reviewed datasets for analysis. Evaluate and recommend improvements in data flow, influencing and supporting architects and engineers. To work independently and to manage multiple data More ❯

Analytics Engineer Location: London (hybrid 3 times a week) Department: IT Type: 3 months rolling contract Outside IR35 The client is seeking an experienced Analytics Engineer to join its Data & Analytics team. The role focuses on building scalable data pipelines, transforming raw data into clean, reusable datasets, and enabling data-driven decision-making. Key Responsibilities Design … and build data products, with proficiency throughout the data lifecycle. Develop robust data models through close collaboration with business users and the engineering team. Partner with senior management, operational leads, and other stakeholders, coaching and supporting a data-driven culture, including KPI definition and reporting frameworks. Accountable for data extraction, transforming JSON & XML, utlising high … experience within metadata management. Collaborate with data engineers to develop data, enrich product design and integrate data for predictive models or machine learning. Deliver well-defined, transformed, tested, documented, and code-reviewed datasets for analysis. Evaluate and recommend improvements in data flow, influencing and supporting architects and engineers. To work independently and to manage multiple data More ❯

Are you passionate about Data and Analytics (D&A) and excited about how it can completely transform the way an enterprise works? Do you have the strategic vision, technical expertise, and leadership skills to drive data-driven solutions? Do you want to work in a dynamic, fast-growing category? If so, you might be the ideal candidate for … the role in the Data and Analytics function for Global Pet Nutrition (PN) at Mars. Pet Nutrition (PN) is the most vibrant category in the FMCG sector. As we work to transform this exciting category, a new program, Digital First, has been mobilized by the Mars Pet Nutrition (PN) leadership team. Digital First places pet parents at the centre … to demonstrate significant thought leadership, quality decision making, deep technical know-how, and an ability to navigate complex business challenges while building and leading a team of world class data and analytics leaders. With Digital First, PN is moving to a Product based model to create business facing digital capabilities. Develop and maintain robust data pipelines and storage More ❯

Are you passionate about Data and Analytics (D&A) and excited about how it can completely transform the way an enterprise works? Do you have the strategic vision, technical expertise, and leadership skills to drive data-driven solutions? Do you want to work in a dynamic, fast-growing category? If so, you might be the ideal candidate for … the role in the Data and Analytics function for Global Pet Nutrition (PN) at Mars. Pet Nutrition (PN) is the most vibrant category in the FMCG sector. As we work to transform this exciting category, a new program, Digital First, has been mobilized by the Mars Pet Nutrition (PN) leadership team. Digital First places pet parents at the centre … to demonstrate significant thought leadership, quality decision making, deep technical know-how, and an ability to navigate complex business challenges while building and leading a team of world class data and analytics leaders. With Digital First, PN is moving to a Product based model to create business facing digital capabilities. Develop and maintain robust data pipelines and storage More ❯

Senior Data Scientists/Analysts – SC/DV Cleared – (Military/Veteran-Friendly) — Multiple Openings 🌳 Location: London (Remote & Hybrid options) 🌳 Employment: Permanent 🌳 Clearance: Current SC or DV required 🌳 Make sense of mission-critical data that actually drives decisions. 🌳 Areti Group is partnering with one of the UK’s fastest-growing Series A-funded tech start-ups to hire … multiple Senior Data Scientists & Data Analysts . You’ll help deliver secure, high-impact analytics platforms for Defence, National Security, and Government projects. 🌳 Perfect for ex-Forces engineers (RAF, Army, Navy) or civilians who’ve worked in high-trust, high-stakes environments and want to use data science at scale . 🌳 What you’ll do 🌳 Build and … deploy machine learning models to solve real-world problems. Design data pipelines, analytics APIs, and decision dashboards used by senior stakeholders. Apply modern data science, AI, and big data techniques to defence and security missions. Deliver secure-by-design, production-grade analytics platforms for sensitive environments. Collaborate across engineering, security, and product teams to deliver at More ❯

Senior Data Scientists/Analysts – SC/DV Cleared – (Military/Veteran-Friendly) — Multiple Openings 🌳 Location: London (Remote & Hybrid options) 🌳 Employment: Permanent 🌳 Clearance: Current SC or DV required 🌳 Make sense of mission-critical data that actually drives decisions. 🌳 Areti Group is partnering with one of the UK’s fastest-growing Series A-funded tech start-ups to hire … multiple Senior Data Scientists & Data Analysts . You’ll help deliver secure, high-impact analytics platforms for Defence, National Security, and Government projects. 🌳 Perfect for ex-Forces engineers (RAF, Army, Navy) or civilians who’ve worked in high-trust, high-stakes environments and want to use data science at scale . 🌳 What you’ll do 🌳 Build and … deploy machine learning models to solve real-world problems. Design data pipelines, analytics APIs, and decision dashboards used by senior stakeholders. Apply modern data science, AI, and big data techniques to defence and security missions. Deliver secure-by-design, production-grade analytics platforms for sensitive environments. Collaborate across engineering, security, and product teams to deliver at More ❯

per day (Inside IR35) Start Date: November 2025 Job Type: 6-month contract Company Introduction: We are looking for a Cloudera Administrator with extensive experience in Cloudera Cloud Data Platform, CDP Services, Terraform and Big data knowledge to join our global client in the finance industry. Job Responsibilities/Objectives: Design and automate the deployment of Cloudera … CDP components (Data Lake, Data Hubs, Data Services) using Terraform . Build and maintain CI/CD pipelines using GitHub Actions for infrastructure and data pipeline automation Collaborate with data engineering teams to integrate CDP with existing Big Data workflows. Write and maintain automation scripts using Shell and Python . Manage infrastructure and … nodes in the cluster. Employ security using an authentication and authorisation system such as Kerberos. Knowledge of changes required for migrating to Cloudera cloud version CDP Knowledge of Cloudera data services (CDW, CDE, CDF, CAI) Design Native Cloud Application Architectures and optimize applications for AWS Network connectivity, Direct Connect, VPN, VPC, Security group, NACL, Route 53 Must have in More ❯

Job Description: Scala/Spark • Good Big Data resource with the below Skillset: Java Big data technologies. • Linux Based Hadoop Ecosystem (HDFS, Impala, Hive, HBase, etc.) • Experience in Big data technologies, real time data processing platform (Spark Streaming) experience would be an advantage. • Consistently demonstrates clear and concise written and verbal communication • A More ❯

Job Description: Scala/Spark • Good Big Data resource with the below Skillset: Java Big data technologies. • Linux Based Hadoop Ecosystem (HDFS, Impala, Hive, HBase, etc.) • Experience in Big data technologies, real time data processing platform (Spark Streaming) experience would be an advantage. • Consistently demonstrates clear and concise written and verbal communication • A More ❯

Databricks Data Engineer £500 - £600 per day Outside IR35 We're partnering with a leading online retail company that's transforming the way data and real-time intelligence shape customer experiences. Their mission is to harness cutting-edge data and streaming technologies to drive smarter decisions, improve efficiency, and create personalised journeys for millions of shoppers worldwide. … The Role As a Senior Data Engineer, you'll play a key role in developing and optimising the backbone of the company's data platform. You'll be responsible for building and maintaining large-scale, real-time data pipelines that power analytics, machine learning, and operational systems across the business. You'll collaborate with software engineers, data scientists, and analytics teams to ensure the platform delivers reliable, high-quality, and compliant data at scale. This is a hands-on engineering role that blends software craftsmanship with data architecture expertise. Key responsibilities: Design and implement high-throughput data streaming solutions using Kafka, Flink, or Confluent. Build and maintain scalable backend systems in Python or More ❯

Who We Are BCG Expand is a specialist firm providing unique data-driven business intelligence as a trusted advisor to senior executives across the world's leading financial services firms to help them to operate more effectively. Expand's activities are focused on data benchmarking in technology, operations, FinTech and market data, with expertise around strategic cost … maintain our professional, supportive and entrepreneurial working culture that we all enjoy. What You'll Do We're looking for a highly skilled Principal Engineer with a passion for data, cloud technologies, and building robust, scalable solutions - who also brings strategic product thinking and cross-functional leadership . In this role, you'll not only lead the design and … implementation of the data infrastructure that powers advanced analytics and data science across our platforms, but also contribute to the strategic direction, roadmap, and execution of data products in collaboration with senior stakeholders. Architect & Build Scalable Data Solutions Collaborate closely with senior product stakeholders to understand data needs and architect end-to-end ingestion pipelines More ❯

DATA SCIENTIST - eDV CLEARED NEW PERMANENT OPPORTUNITY AVAILABLE FOR A DATA SCIENTIST IN LONDON WITH ENHANCED DV CLEARANCE Permanent opportunity for a Data Scientist Enhanced DV security clearance is required Salary up to £95,000 + Clearance Bonus + Company Bonus Central London based organisation in an accessible location To apply please call/or email WHO … WE ARE? We are recruiting a permanent Data Scientist to work with a leading technology and engineering company that do excellent work in the National Security sector across a range of exciting projects. Due to the nature of our clients, you must hold enhanced DV security clearance. WHAT WILL THE DATA SCIENCE ENGINEER BE DOING? As a Data … you will be working alongside our team on a wide variety of projects with a special focus on innovation and AI. You could be working with all types of data (both structured and unstructured) using a variety of software - so we're open to seeing engineers with different technology backgrounds! THE DATA SCIENCE ENGINEER SHOULD HAVE.... Strong and More ❯

Fenestra provides the largest advertisers and agencies worldwide with a customizable platform and data ecosystem that empowers programmatic buyers to understand and automatically optimize their advertising investments globally. Founded by proven digital advertising executives, rapid growth through our SOTA automated campaign optimization technology (“Actus”) built on our rich analytics platform leads us to seek new talent to lead us … level of automated bidding. We seek an exceptional technologist to serve as our Head of Engineering, reporting directly to the CEO and leading our team of internal and external Data Engineers, Data Analysts, and Data Scientists. This hands-on role will shape our technical vision, drive engineering excellence, and embody Fenestra's technical capabilities. You'll have … deep technical expertise to drive our data-powered automation solutions forward. Key Responsibilities Technical Leadership & Architecture Define and execute engineering strategy aligned with business objectives Develop and lead a high-performing team of Data Engineers, Data Analysts, and Data Scientists Architect scalable, high-performance data infrastructure with sophisticated system design principles Apply advanced computer science More ❯

Role: Cloudera Admin Location: Stratford, UK Job Type: Contract Work Mode: Hybrid (3 days onsite a week) Cloudera Admin: Design and automate the deployment of Cloudera CDP components (Data Lake, Data Hubs, Data Services) using Terraform. Build and maintain CI/CD pipelines using GitHub Actions for infrastructure and data pipeline automation Collaborate with data engineering teams to integrate CDP with existing Big Data workflows. Write and maintain automation scripts using Shell and Python. Manage infrastructure and configuration using YAML and JSON. Configuring and tuning Cloudera services for performance and security. Ensure secure, scalable, and cost-effective deployments on cloud platforms (AWS, Azure, or GCP). An understanding of all the Hadoop … nodes in the cluster. Employ security using an authentication and authorization system such as Kerberos. Knowledge of changes required for migrating to Cloudera cloud version CDP Knowledge of Cloudera data services (CDW, CDE, CDF, CAI) Design Native Cloud Application Architectures and optimize applications for AWS Network connectivity, Direct Connect, VPN, VPC, Security group, NACL, Route 53 Must have in More ❯

Role: Cloudera Admin Location: Stratford, UK Job Type: Contract Work Mode: Hybrid (3 days onsite a week) Cloudera Admin: Design and automate the deployment of Cloudera CDP components (Data Lake, Data Hubs, Data Services) using Terraform. Build and maintain CI/CD pipelines using GitHub Actions for infrastructure and data pipeline automation Collaborate with data engineering teams to integrate CDP with existing Big Data workflows. Write and maintain automation scripts using Shell and Python. Manage infrastructure and configuration using YAML and JSON. Configuring and tuning Cloudera services for performance and security. Ensure secure, scalable, and cost-effective deployments on cloud platforms (AWS, Azure, or GCP). An understanding of all the Hadoop … nodes in the cluster. Employ security using an authentication and authorization system such as Kerberos. Knowledge of changes required for migrating to Cloudera cloud version CDP Knowledge of Cloudera data services (CDW, CDE, CDF, CAI) Design Native Cloud Application Architectures and optimize applications for AWS Network connectivity, Direct Connect, VPN, VPC, Security group, NACL, Route 53 Must have in More ❯