1 to 25 of 70 Big Data Jobs in London

BIG DATA ENGINEER - DV CLEARED NEW PERMANENT JOB OPPORTUNITY AVAILABLE WITHIN A LEADING NATIONAL SECURITY SME FOR A BIG DATA ENGINEER WITH DV CLEARANCE Permanent job opportunity for a Big Data Engineer Leading National Security & Defence SME Salary up to £80,000 plus clearance bonus London based organisation in an easily accessible location To … apply please call or email WHO WE ARE? We are recruiting multiple Big Data Engineers to support urgent National Security & Defence projects in London. Due to the nature of these projects you must hold DV or enhanced DV Security Clearance. WHAT WILL THE BIG DATA ENGINEER BE DOING? You will be joining a leading SME who … is working hard to support National Security projects within UK Govt. Departments in London. As part of a team, you will be responsible for implementing Big Data Solutions in Mission-Critical areas. WE NEED THE BIG DATA ENGINEER TO HAVE.... Current DV clearance - Standard or Enhanced Must have experience with big data tools such More ❯

company that solves real-world problems for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on … We're underpinned by over 300 engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data architectures such as Data Mesh, lakehouse, data vault and data warehouses. Our data engineers create pipelines that support our data scientists and power our front-end applications. This means we do data-intensive work for both OLTP and OLAP use cases. Our environments are primarily cloud-native spanning AWS, Azure and GCP, but we also work on systems More ❯

offering. Delivering our strategy sustainably is fundamental to achieving our ambition to be a net zero company by 2050 or sooner! About the Role : We are seeking a Senior Data Engineer to join our growing data engineering team. In this role, you will be responsible for designing, building, and optimizing scalable data pipelines and infrastructure that power … analytics, machine learning, and business intelligence across the organization. You will work closely with data scientists, analysts, and platform engineers to ensure robust and efficient data delivery. This is an excellent opportunity for someone who is passionate about data, cloud technologies, and scalable systems, and wants to have a direct impact on data-driven decision-making. … Key Responsibilities: Design, build, and maintain scalable and reliable ETL/ELT data pipelines using Python, PySpark, and SQL. Develop and manage data workflows and orchestration using tools such as Airflow or similar. Optimize data processes for performance, scalability, and cost-efficiency, particularly in cloud environments. Collaborate with data scientists, analysts, and product teams to understand More ❯

CONTRACT DATA ENGINEER - eDV CLEARED NEW OUTSIDE IR35 CONTRACT OPPORTUNITY AVAILABLE WITHIN A LEADING NATIONAL SECURITY SME FOR A DATA ENGINEER WITH eDV CLEARANCE Contract job opportunity for a Data Engineer National Security client Palantir Foundry Outside IR35 Central London based organisation in an easily accessible location To apply please call/or email WHO WE ARE … We are recruiting a contract Data Engineer to work with a National Security SME in central London. Due to the nature of the work, you must hold enhanced DV Clearance. WE NEED THE DATA ENGINEER TO HAVE.... Current enhanced DV Security Clearance Experience with big data tools such as Hadoop, Cloudera or Elasticsearch Experience With Palantir … Scrum environment with tools such as Confluence/Jira Experience in design, development, test and integration of software Willingness to learn new technologies IT WOULD BE NICE FOR THE DATA ENGINEER TO HAVE.... Cloud based architectures Microservice architecture or server-less architecture Messaging/routing technologies such as Apache Nifi/RabbitMQ TO BE CONSIDERED.... Please either apply by More ❯

CONTRACT PYTHON/DATA ENGINEER - DV CLEARED NEW OUTSIDE IR35 CONTRACT OPPORTUNITY AVAILABLE WITHIN A LEADING NATIONAL SECURITY SME FOR A PYTHON/DATA ENGINEER WITH DV CLEARANCE Contract job opportunity for a Data Engineer National Security & Defence client Outside IR35 12 month rolling contract Central London based organisation in an easily accessible location To apply please … call/or email WHO WE ARE? We are recruiting a contract Python/Data Engineer to work with a National Security & Defence SME in central London. Due to the nature of the work, you must hold UKSV/MOD or Enhanced DV Clearance. WE NEED THE PYTHON/DATA ENGINEER TO HAVE.... Current DV Security Clearance (Standard … or Enhanced) Experience with big data tools such as Hadoop, Cloudera or Elasticsearch Python/PySpark experience Experience With Palantir Foundry is nice to have Experience working in an Agile Scrum environment with tools such as Confluence/Jira Experience in design, development, test and integration of software Willingness to learn new technologies TO BE CONSIDERED.... Please either More ❯

DATA ARCHITECT - DV CLEARED NEW PERMANENT JOB OPPORTUNITY AVAILABLE WITHIN A LEADING NATIONAL SECURITY SME FOR A DATA ARCHITECT WITH ENHANCED DV CLEARANCE Permanent job opportunity for a Data Architect Leading National Security SME Salary up to £100,000 + Bonus London based organisation in an easily accessible location To apply please call or email WHO WE … ARE? We are recruiting multiple a Data Architect to support urgent National Security & Defence projects in London. Due to the nature of these projects you must hold enhanced DV Security Clearance. WHAT WILL THE DATA ARCHITECT BE DOING? You will be joining a leading SME who is working hard to support National Security projects within UK Govt. Departments … in London. As part of a team, you will be responsible for designing and implementing Data Solutions in Mission-Critical areas. WE NEED THE DATA ARCHITECT TO HAVE.... Current DV clearance - Enhanced Good at understanding complexity and abstracting that into a form that is consumable for a non-technical audience. Experience of achieving data interoperability between More ❯

Requisition ID 10803 -Posted - Years of Experience (1) - Technology - Where (1) - Job Data Reply is the Reply Group company offering a broad range of analytics and data processing services. We operate across different industries and business functions, working directly with executive level professionals, enabling them to achieve meaningful outcomes through effective use of data. We find that one … of the biggest problems experienced by our clients today is being overwhelmed with the amount of data that they face and not knowing how to leverage it to their advantage. The vast landscape of available technology stacks and models means that choosing the right ones can be a daunting task. Most companies know that their data is valuable … and that they should be making the most out of it to stay competitive, but often don't know where to begin or what to prioritise. At Data Reply, we pride ourselves on helping clients make the right decisions to build their data strategy. With our consultants' expertise, we map the right technologies to meet our clients' business More ❯

Overview stackable - who are we? Stackable is an innovative technology company focused on delivering cutting-edge, opensource Big Data solutions for data lakehouses, data mesh, event processing and AI. We specialise in designing, deploying, and managing scalable datainfrastructures that empower businesses to harness the power of data effectively. Our core product, the Stackable Data Platform, enables companies of all sizes to easily integrate and manage their sovereign data platforms in their respective IT infrastructure. Your mission at stackable: We are looking for a skilled Solutions Architect with a strong background in Big Data to join our team. The ideal candidate will have expertise in open source Big Data technologies and Kubernetes and DevOps practices . This role will be instrumental in designing, implementing, and optimizing cloud-based and on-premise data solutions for our clients. Responsibilities You will architect and implement Stackable Data Platform solutions , ensuring scalability, reliability, and security. You will develop and maintain Kubernetes-based infrastructures to support data processing and analytics More ❯

Lead data engineers at Thoughtworks develop modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. They might spend a few weeks with a new client on a deep technical review or a complete organizational review, helping them to understand the potential that data brings to solve their most pressing … overseeing a program inception to build a new product. Alongside hands-on coding, they are leading the team to implement the solution. Job responsibilities You will lead and manage data engineering projects from inception to completion, including goal-setting, scope definition and ensuring on-time delivery with cross team collaboration. You will collaborate with stakeholders to understand their strategic … objectives and identify opportunities to leverage data and data quality. You will design, develop and operate modern data architecture approaches to meet key business objectives and provide end-to-end data solutions. You will be responsible to create, design and develop intricate data processing pipelines, addressing clients' most challenging problems. You will collaborate with data More ❯

A global IT services, consulting, and business solutions company has partnered with a leading financial services organisation to hire an experienced Data Architect for a 6-month contract. In this role, you'll design and implement enterprise-scale Data Lake solutions, define architectural standards, and guide best practices across large-scale digital transformation programmes. You'll collaborate with … cross-functional teams to ensure scalable, secure, and high-performance data systems that align with business objectives. This is a hybrid role requiring 2 days per week on-site in Central London. Ready to shape the future of data architecture in financial services? Apply now to join a high-impact team at the forefront of transformation. Key Responsibilities … Design and develop enterprise-grade data architecture frameworks supporting Data Lakes and Big Data platforms Create conceptual, logical, and physical data models for robust, scalable storage solutions Define and maintain the overall structure of data flows and models across multiple platforms Oversee integration of diverse data sources into unified and secure data More ❯

Job ID: Amazon Web Services Australia Pty Ltd Are you a Senior Data Analytics and GenAI consulting specialist? Do you have real-time Data Analytics, Data Warehousing, Big Data, Modern Data Strategy, Data Lake, Data Engineering and GenAI experience? Do you have senior stakeholder engagement experience to support pre-sales and deliver … consulting engagements? Do you like to solve the most complex and high scale (billions+ records) data challenges in the world today? Do you like leading teams through high impact projects that use the latest data analytics technologies? Would you like a career path that enables you to progress with the rapid adoption of cloud computing? AWS Professional Services … Public Sector ANZ are hiring a highly technical senior cloud architect specialised in Data Analytics and GenAI to collaborate with our customers and partners to derive business value from the latest Data Analytics and GenAI services. Our consultants will develop and deliver proof-of-concept projects, technical workshops and support complex projects. These professional services engagements will focus More ❯

Who We Are BCG Expand is a specialist firm providing unique data-driven business intelligence as a trusted advisor to senior executives across the world's leading financial services firms to help them to operate more effectively. Expand's activities are focused on data benchmarking in technology, operations, FinTech and market data, with expertise around strategic cost … maintain our professional, supportive and entrepreneurial working culture that we all enjoy. What You'll Do We're looking for a highly skilled Principal Engineer with a passion for data, cloud technologies, and building robust, scalable solutions - who also brings strategic product thinking and cross-functional leadership . In this role, you'll not only lead the design and … implementation of the data infrastructure that powers advanced analytics and data science across our platforms, but also contribute to the strategic direction, roadmap, and execution of data products in collaboration with senior stakeholders. Architect & Build Scalable Data Solutions Collaborate closely with senior product stakeholders to understand data needs and architect end-to-end ingestion pipelines More ❯

DATA SCIENTIST - eDV CLEARED NEW PERMANENT OPPORTUNITY AVAILABLE FOR A DATA SCIENTIST IN LONDON WITH ENHANCED DV CLEARANCE Permanent opportunity for a Data Scientist Enhanced DV security clearance is required Salary up to £95,000 + Clearance Bonus + Company Bonus Central London based organisation in an accessible location To apply please call/or email WHO … WE ARE? We are recruiting a permanent Data Scientist to work with a leading technology and engineering company that do excellent work in the National Security sector across a range of exciting projects. Due to the nature of our clients, you must hold enhanced DV security clearance. WHAT WILL THE DATA SCIENCE ENGINEER BE DOING? As a Data … you will be working alongside our team on a wide variety of projects with a special focus on innovation and AI. You could be working with all types of data (both structured and unstructured) using a variety of software - so we're open to seeing engineers with different technology backgrounds! THE DATA SCIENCE ENGINEER SHOULD HAVE.... Strong and More ❯

Lead Data Scientist LondonPrivate Equity£140,000 - £150,000 + benefits Harnham is working with a tech firm in the Private Equity space and hiring a few positions across Data Science, Machine Learning Engineering and Data Engineering. The company have an impressive founding team and hold a strong reputation within the Data & AI and Private Equity … purpose of creating value. Often working in ambiguous and greenfield environments, you will collaborate with a strong team of technical people across multiple industries to solve complex commercial and data problems, delivering key data science initiatives that drive performance and revenue. In this role you will lead the technical teams that deliver value. The Role - Lead Data Scientist You will be: Working on multiple engagements within portfolio companies across various different industry sectors Delivering key Data Science/AI initiatives that drive value for customers. Shaping how projects are delivered, driving performance and working closely with C-level execs. Providing technical leadership across the business, with a focus on end-to-end machine learning solutions. More ❯

Qualifications Bachelor's or Master's degree in Computer Science, Mathematics or similar field (PhDs will be given preference) 7+ years of experience working with massive data Strong knowledge of public cloud (AWS) Experience with data acquisition (API calls/FTP downloads), ETL, transformation/normalization Proficiency in ETL process is mandatory Experienced building components for enterprise central … data platforms (data warehouses, Operational Data Stores, Access layers with APIs, file extracts, user queries) Hands-on experience with SQL, Python, Spark, Kafka Excellent communication skills, including a good working proficiency in verbal and written English About this Job Scopeworker Business Intelligence real time processes enterprise customer data. We develop and maintain robust and reliable data pipelines to support 24x7 business operations and enable systems, tools, dashboards and data analysts to surface business insights and opportunities. This actionable business intelligence is provided in real time to our Fortune 100 users from data engineers, to analysts and C-Level stakeholders. As a Data Architect, you'll innovate, design, build and automate, scalable solutions More ❯

Senior Data Engineer Hybrid working - 1 day a week onsite in either London or Portsmouth Permanent - Up to £75k Overview: We are seeking a talented Senior Data Engineer specialising in Starburst (Trino) and Dell Data Lakehouse to join our AI & Data team. You will be responsible for deploying, maintaining and optimising Starburst installations & Dell Data Lakehouse, enabling our clients to seamlessly access their data across multiple platforms. The ideal candidate will have excellent communication skills, an advanced understanding of Starburst & Dell Data Lakehouse, and proficiency with troubleshooting and root cause analysis. Responsibilities: Deploy and manage Starburst Enterprise/Galaxy and Dell Data Lakehouse installations, overseeing environment setup, configuration, maintenance, upgrades, and … ensuring optimal performance. Configure various server and application settings and parameters. Integrate Starburst with various data sources to create a unified data platform. Design and tune the container solution for performance and scalability. Set up and configure data catalogs in various modes. Implement robust security controls for data access, ensure compliance with data regulations, and More ❯

Senior Data Engineer Hybrid working - 1 day a week onsite in either London or Portsmouth Permanent - Up to £75k Overview: We are seeking a talented Senior Data Engineer specialising in Starburst (Trino) and Dell Data Lakehouse to join our AI & Data team. You will be responsible for deploying, maintaining and optimising Starburst installations & Dell Data Lakehouse, enabling our clients to seamlessly access their data across multiple platforms. The ideal candidate will have excellent communication skills, an advanced understanding of Starburst & Dell Data Lakehouse, and proficiency with troubleshooting and root cause analysis. Responsibilities: Deploy and manage Starburst Enterprise/Galaxy and Dell Data Lakehouse installations, overseeing environment setup, configuration, maintenance, upgrades, and … ensuring optimal performance. Configure various server and application settings and parameters. Integrate Starburst with various data sources to create a unified data platform. Design and tune the container solution for performance and scalability. Set up and configure data catalogs in various modes. Implement robust security controls for data access, ensure compliance with data regulations, and More ❯

Join Pay.UK as a Data Scientist and contribute to the enhancement of our data science, machine learning, AI, and predictive analytics platforms. You'll build predictive models, provide quantitative analysis, and uncover trends using advanced techniques. Your role will involve managing strategic insights, understanding the payments landscape, and ensuring transparency in the supplier market. You'll also apply … data strategies, including GDPR compliance, and analyse market scenarios to support business decisions. This is a unique opportunity to innovate and mitigate fraud in the payments industry. Key Accountabilities: Work with Big Data technologies to shape the development of analytical insight delivery Use statistical data approaches to uncover patterns in data from which predictive models … can be developed Identify valuable data sources and automate collection processes. Apply statistical techniques on large datasets to: Measure results and identify key underlying trends Build hypothesis and test significance Identify predictive indicators and causal impact and attribution Build predictive models to identify the likelihood of fraud Analyse geo-demographic and socio-economic profiling data along with transactional More ❯

Join an IT Consultancy that has grown from £200m in revenue in 2020 to over £1b in 2024... Data Architect £85,000 basic Central London (Hybrid) Your new opportunity... As a Data Architect, you’ll be responsible for designing, implementing, and optimizing data architectures that empower clients to unlock the value of your customers data. You’ll … be responsible for: Designing and delivering data architectures that integrate multiple data sources, including SQL and NoSQL databases. Implementing and optimizing data warehouse solutions and ETL/ELT pipelines for analytics and reporting. Working with big data ecosystems such as Hadoop, Spark, and Kafka to build scalable solutions. What you’ll bring... Strong expertise in … SQL and NoSQL technologies, such as Oracle, PostgreSQL, MongoDB, or similar. Proven experience with data warehousing concepts and ETL/ELT tools. Knowledge of big data platforms and streaming tools like Hadoop, Spark, and Kafka. A deep understanding of scalable data architectures, including high availability and fault tolerance. Experience working across hybrid or cloud environments. Excellent More ❯

Job Description: Our tech stack AWS is a significant growth area for us with a diverse and growing capability and we are looking for a Data Engineer with experience in AWS cloud technologies for ETL pipeline, data warehouse and data lake design/building and data movement. AWS data and analytics services (or open-source … Essential requirements You thrive in problem-solving and analytical thinking You enjoy collaborating with multiple stakeholders in a fast-paced environment Experience in the design and deployment of production data pipelines from ingestion to consumption within a big data architecture, using Java, Python, Scala, Spark, SQL. Experience performing tasks such as writing scripts, extracting data using … APIs, writing SQL queries etc. Experience in processing large amounts of structured and unstructured data, including integrating data from multiple sources through ingestion and curation functions on AWS cloud using AWS native or custom programming More ❯

Reports to: Team Leader, Analytics, Claims and Underwriting Solutions (International) Location: London/Hybrid Business Description Verisk Analytics is a multinational data and analytics risk assessment company and a leading source of information about insurance risk. To serve our clients, we draw upon our vast experience in data management and predictive modelling to offer decision analytics to the … residential and commercial property perils, motor insurance, SME business covers, travel, pet and health insurance. To feed our risk models we source many diverse datasets covering high resolution weather data, property attributes, business activities and financial data, insurance claims and policies, medical information, aerial imagery and geodemographic data. In claims solutions we support end-to-end claims handling … real-time fraud analysis and image forensics we help detect fraud at every stage of the claim and manage cases more effectively. Role Purpose Verisk are looking for a Data Scientist to support the development of various data science models to support the development of our risk management products. The International Analytics Team is a dynamic and exciting More ❯

and hands-on experience and solid understanding of machine learning and deep learning methods • PhD in a quantitative discipline, e.g. Computer Science, Electrical Engineering, Mathematics, Operations Research, Optimization, or Data Science with reasonable industry experience, or an MS with significant industry or research experience in the field • Extensive experience with machine learning and deep learning toolkits (e.g.: TensorFlow, PyTorch … NumPy, Scikit-Learn, Pandas) • Ability to design experiments and training frameworks, and to outline and evaluate intrinsic and extrinsic metrics for model performance aligned with business goals • Experience with big data and scalable model training and solid written and spoken communication to effectively communicate technical concepts and results to both technical and business audiences. • Scientific thinking with the … the financial services industries and continuous integration models and unit test development • Knowledge in search/ranking, Reinforcement Learning or Meta Learning • Experience with A/B experimentation and data/metric-driven product development, cloud-native deployment in a large scale distributed environment and ability to develop and debug production-quality code • Published research in areas of Machine More ❯

The Chief Data & Analytics Office (CDAO) at JPMorgan Chase is responsible for accelerating the firm's data and analytics journey. This includes ensuring the quality, integrity, and security of the company's data, as well as leveraging this data to generate insights and drive decision-making. The CDAO is also responsible for developing and implementing solutions … machine learning technologies to develop new products, improve productivity, and enhance risk management effectively and responsibly.As an Applied AI ML Director - NLP/LLM and Graphs within the Chief Data & Analytics Office, Machine Learning Centre of Excellence, you will have the opportunity to apply sophisticated machine learning methods to complex tasks including natural language processing, graph analytics, speech analytics … application of machine learning models across different areas of the businessRequired qualifications, capabilities, and skillsPhD in a quantitative discipline, e.g. Computer Science, Electrical Engineering, Mathematics, Operations Research, Optimization, or Data Science Or an MS with significant years of industry or research experience in the field.Solid background in NLP, LLM and graph analytics and hands-on experience and solid understanding More ❯

Principal Cybersecurity Machine Learning Engineer The position is a senior technical role, requiring knowledge of data engineering, data science, advanced analytics techniques, research and development, and innovation within one or more of the Cybersecurity sciences (e.g. network, endpoint and cloud security, cryptography, malware analysis, vulnerability assessment, privileged access management, anomaly/intrusion detection, incident response, or offensive security … based employees, enhanced maternity and adoption pay and support when you return to work, and a contributory pension scheme with a generous employer contribution. In this role you will: Data Engineering Ingestion and provisioning of datasets, delta tables, and/or curated, re-usable data assets to enable Cybersecurity use cases. Support and enhancement of data ingestion … Research & Development (R&D) and Innovation Drive innovative research & development to test and evolve new technologies and advanced analytic techniques. Explore and develo innovative capabilities, exploiting emerging technologies and data science to help stay ahead of advancing cyber threats. Rapid Response Respond to urgent requests to address vulnerabilities or emerging threats. Advanced Analytics & Security Software Development Support Cybersecurity teams More ❯

Databricks Data Architect - London Join a dynamic team committed to data excellence and innovation. We are seeking a talented Data Architect to join our client's Azure - Data team in London. **Key Responsibilities:**- Design and implement data architecture using Azure and Databricks Lakehouse.- Collaborate with cross-functional teams to drive data strategy and governance. … Ensure data integrity and security within cloud environments.- Contribute to the development of data models and pipelines for analysis and reporting.**Required Skills:**- Strong experience with Azure cloud services and infrastructure.- Proficiency in Databricks Lakehouse architecture.- Knowledge of data management practices and big data technologies.- Excellent problem-solving and analytical skills.This role is perfect for … candidates looking to make a significant impact within a supportive and innovative team. If you are passionate about leveraging data to drive business results, we encourage you to apply for this opportunity. More ❯