51 to 75 of 147 Data Lake Jobs in London

QUALIFICATIONS - 5+ years of experience in cloud architecture - 7+ years track record of consulting, managing and delivering large-scale enterprise implementations - Experience designing and implementing cloud solutions, including data analytics and ML components - Bachelor's degree in Computer Science, Engineering, or related field - Able to communicate effectively in English within technical and business settings. PREFERRED QUALIFICATIONS - Advanced technology … Data & Analytics) - Deep understanding of AWS services across compute, storage, networking, security, databases, machine learning, and serverless technologies - Experience defining data architectures (Data Lake, Lake House, Data Mesh, etc) and ML projects in cloud environments - Expertise in performance optimization and cost management for cloud environments - Ability to manage multiple projects … and priorities in a fast-paced environment - Working knowledge of common data processing tools and ML frameworks - AWS Professional level certifications (e.g., Solutions Architect Professional, DevOps Engineer Professional) preferred Our inclusive culture empowers Amazonians to deliver the best results for our customers. If you have a disability and need a workplace accommodation or adjustment during the application and More ❯

to work in a company where people truly BELIEVE in what they are doing! We're committed to bringing passion and customer focus to the business. Solution Architect (Data Engineering) London Fractal is a strategic AI partner to Fortune 500 companies with a vision to power every human decision in the enterprise. Fractal is building a world where … development experience Proficient in Object-oriented languages (Python, PySpark) and frameworks Hands-on expertise in Azure ecosystem, including components like Azure Data Factory, Azure Data Lake Storage, Azure, SQL, Azure DataBricks, HD Insights, ML Service etc. Expertise in relational and dimensional modelling, including big data technologies. Experience in Azure Infrastructure and Azure Dev … Mandatory Skills : Proficient in Object-oriented languages (Python, PySpark) and frameworks Hands-on expertise in Azure ecosystem, including components like Azure Data Factory, Azure Data Lake Storage, Azure, SQL, Azure DataBricks, HD Insights, ML Service etc. Expertise in relational and dimensional modelling, including big data technologies. Experience in Azure Infrastructure and Azure Dev More ❯

Social network you want to login/join with: Senior Data Engineering Consultant, London Client: Location: London, United Kingdom Job Category: Other EU work permit required: Yes Job Reference: d39bd431c836 Job Views: 4 Posted: 29.06.2025 Expiry Date: 13.08.2025 Job Description: Our client, a global leader in the technology and consulting industry, is seeking a visionary senior data engineering consultant to drive their digital transformation. This is a pivotal role in shaping the future of the business through data-driven innovation. The Challenge: Architect and implement a world-class data platform to support business growth and competitive advantage. Lead complex data initiatives, delivering measurable business impact. Build a high-performing … data team capable of executing ambitious data strategies. The Opportunity: This is an exceptional opportunity to join a forward-thinking organisation at the forefront of data-driven innovation. You will be instrumental in shaping our data strategy and delivering transformative solutions that drive business growth. Lead complex, high-impact data More ❯

powering our marketing technology and performance efforts. As a Junior MarTech Analyst, you will be embedded within the acquisition and performance marketing teams, working closely with Marketing Managers, Data Engineers, and MarTech Operations leads. You'll collaborate with cross-functional partners wider data teams to connect, segment, and optimise data flows that drive our … advertising platforms. This role offers a great opportunity to develop your technical skills, particularly in SQL and data engineering, while contributing directly to marketing growth and innovation. You'll have the autonomy to support and improve our marketing data infrastructure, helping the team make data-driven decisions that fuel Trainline's success. As a … our data platforms and advertising channels. You'll collaborate with stakeholders to: Connect and configure customer, route, and pricing data from our data lake into marketing platforms via rETL and feed management platforms like Census and Channable Segment and manipulate data to ensure it is in the correct format for ad More ❯

Working at GSR is an opportunity to be deeply embedded in every major sector of the cryptocurrency ecosystem. About the role: This role sits within GSR's global Data Engineering team, where you'll contribute to the design and development of scalable data systems that support our trading and business operations. You'll work closely with … stakeholders across the firm to build and maintain pipelines, manage data infrastructure, and ensure data is reliable, accessible, and secure. It's a hands-on engineering position with scope to shape the way data is handled across the business, working with modern tools in a fast-moving, high-performance environment. Your responsibilities may include … an equivalent framework such as Spark Structured Streaming or Kafka Streams)? Select Which statement best describes your hands on responsibility for architecting and tuning cloud native data lake/warehouse solutions (e.g., AWS S3 + Glue/Redshift, GCP BigQuery, Azure Synapse)? Select What best reflects your experience building ETL/ELT workflows with Apache Airflow (or More ❯

manufacturing supply chains robust to an ever-changing world, and to harness the potential of generative AI through thoughtful deployment, maximising benefits while prioritising ethical use and safety. Data Engineers are the backbone of Magentic. They design, build, and maintain the scalable data infrastructure that powers our intelligent agentic systems. You'll work closely with ML … engineers, product teams, and domain experts to transform raw supply chain data into structured, reliable, and actionable insights that drive decision-making across our platform. We're creating the data foundation for autonomous agents to reason, plan, and act in complex, high-stakes environments. This role is an opportunity to work on deeply technical challenges with … years building and running data pipelines in AWS or Azure, including managed data services (e.g., Kinesis, EMR/Databricks, Redshift, Glue, Azure Data Lake). Programming Mastery: Advanced skills in Python or another major language; writing clean, testable, production-grade ETL code at scale. Modern Data Pipelines: Experience with batch and More ❯

the largest energy supplier in the UK. This growth has been possible due to our custom technology, innovative products and obsession with delighting our customers. Our Industry Operations Data team plays a crucial role in managing internal key data, such as Billing, Metering and Dataflows as well as the data exchanged with industry bodies … software engineering best practices to design, test, and deploy our data platform and services using the below technologies: Python as our main programming language Databricks as our datalake platform Kubernetes for data services and task orchestration Terraform for infrastructure Streamlit for data applications Airflow purely for job scheduling and tracking Circle CI for continuous … deployment Parquet and Delta file formats on S3 for data lake storage Spark for data processing DBT for data modelling SparkSQL for analytics Why else you'll love it here Wondering what the salary for this role is? Just ask us! On a call with one of our recruiters it's something we More ❯

Join Barclays as a Digital Customer Data Engineer, where you'll be leading the development for a new shared data platform that is transforming how individuals/organisations interact with the bank. Your role involves using the latest data and AI engineering skills to deliver a digital capability. As part of our team, you … ll also benefit from being part of a vast professional network, collaborating with industry mentors and experts. To be successful you should have experience with: Modern data engineering practices, inc. event-driven and data lake. Digital platform development, inc. experience developing colleague and customer facing applications. Banking software development. Deep technical awareness of cloud and data technology, such as Kafka, Data Bricks, microservices, API. Some other highly valued skills may include: Technical design: Analysis and documentation of system characteristics, inc. designing solutions to meet multi-faceted non-functional requirements. Communication Skills: Ability to clearly convey project goals to senior leadership, updates, and technical details to stakeholders at all levels, including non-technical More ❯

Join Barclays as a Digital Customer Data Engineer, where you'll be leading the development for a new shared data platform that is transforming how individuals/organisations interact with the bank. Your role involves using the latest data and AI engineering skills to deliver a digital capability. As part of our team, you … ll also benefit from being part of a vast professional network, collaborating with industry mentors and experts. To be successful you should have experience with: Modern data engineering practices, inc. event-driven and data lake. Digital platform development, inc. experience developing colleague and customer facing applications. Banking software development. Deep technical awareness of cloud and data technology, such as Kafka, Data Bricks, microservices, API. Some other highly valued skills may include: Technical design: Analysis and documentation of system characteristics, inc. designing solutions to meet multi-faceted non-functional requirements. Communication Skills: Ability to clearly convey project goals to senior leadership, updates, and technical details to stakeholders at all levels, including non-technical More ❯

of millions of policies and raised over $200 million in funding. And we're only just getting started. About the Role We're looking for a Head of Data Engineering to lead and grow our high-impact team of four Data Engineers. You'll play a pivotal role in shaping the future of Zego's data infrastructure- ensuring it scales with our ambitions while remaining resilient, efficient, and safeguarding user data privacy. This is a hands-on leadership role that combines technical depth, strategic vision, and team enablement. You'll partner with cross-functional leaders in Engineering, Data Science, Analytics, and Product to drive our data strategy, evolve … feedback, and knowledge sharing. Pragmatic Problem-Solving: Comfortable balancing technical excellence with business priorities and constraints Nice to have Experience building Data Mesh or Data Lake architectures. Familiarity with Kubernetes, Docker, and real-time streaming (e.g. Kafka, Kinesis). Exposure to ML engineering pipelines or MLOps frameworks. What's it like to work at Zego More ❯

the Kraken technology platform, designed from the ground up for the energy sector, and covering everything from billing to managing customer interactions. We are looking for an experienced data leader to lead the data analysts & scientists supporting our operational teams, enabling them to extract key actionable insights from the Kraken platform, and help the teams to … software engineering best practices to design, test, and deploy our data platform and services using the below technologies: Python as our main programming language Databricks as our datalake platform Kubernetes for data services and task orchestration Terraform for infrastructure Streamlit for data applications Airflow purely for job scheduling and tracking Circle CI for continuous … deployment Parquet and Delta file formats on S3 for data lake storage Spark for data processing DBT for data modelling SparkSQL for analytics Why else you'll love it here Wondering what the salary for this role is? Just ask us! On a call with one of our recruiters it's something we More ❯

Location: London, England, United Kingdom Join Axon and be a Force for Good. Design, build, and maintain scalable, secure, and efficient data pipelines using modern data tools and cloud-native technologies Work with a variety of data sources-structured, semi-structured, unstructured-to support diverse analytics and operational workloads Ensure data quality … and consistency by incorporating automated testing, data validation, and monitoring mechanisms Drive best practices around data engineering, including testing, observability, security, and documentation Troubleshoot and resolve issues in production environments to ensure data integrity and platform reliability What you bring: 5+ years of professional experience in data engineering roles, preferably for a … effective use of AI tooling in your development process A growth mindset and eagerness to work in a fast-paced, mission-driven environment Good understanding of data lake architecture, medallion architecture, and data warehouse concepts. Good understanding of cloud architectures and technologies, including Docker and Kubernetes Good data modelling skills, and experience applying More ❯

Data Architect Location: Hybrid - London (2-3 days onsite) Type: Permanent Industry: Data Consultancy Salary: Competitive + Benefits The Opportunity Our client, a growing data consultancy with deep expertise in the Microsoft Azure ecosystem, is looking for a Data Architect to join their high-performing team. This role is both strategic and … hands-on, focused on designing modern, scalable, and secure data platforms for enterprise clients. You'll play a key role in shaping data architecture across the full Azure stack- including Azure Databricks and Azure Data Factory (ADF) -and will guide engineering teams in delivering robust, future-proof solutions using … lakehouse and medallion architecture principles . Key Responsibilities Design end-to-end data architectures using Azure services, including Azure Databricks, ADF, Synapse Analytics , and Data Lake Storage Define scalable data models and implement architectural patterns such as lakehouse and medallion Lead technical solution design during client engagements, from discovery to delivery Establish and More ❯

Data Architect Location: Hybrid - London (2-3 days onsite) Type: Permanent Industry: Data Consultancy Salary: Competitive + Benefits The Opportunity Our client, a growing data consultancy with deep expertise in the Microsoft Azure ecosystem, is looking for a Data Architect to join their high-performing team. This role is both strategic and … hands-on, focused on designing modern, scalable, and secure data platforms for enterprise clients. You'll play a key role in shaping data architecture across the full Azure stack- including Azure Databricks and Azure Data Factory (ADF) -and will guide engineering teams in delivering robust, future-proof solutions using … lakehouse and medallion architecture principles . Key Responsibilities Design end-to-end data architectures using Azure services, including Azure Databricks, ADF, Synapse Analytics , and Data Lake Storage Define scalable data models and implement architectural patterns such as lakehouse and medallion Lead technical solution design during client engagements, from discovery to delivery Establish and More ❯

remote Benefits: Healthcare and great holidays The Role As a Machine Learning Operations Engineer, you will be instrumental in designing, deploying, and maintaining machine learning systems that drive data-driven decision-making in the pensions and financial services sector. Using Azure's cloud ecosystem, you will build scalable ML workflows that support actuarial modelling, risk prediction, and client … regression, ensemble methods, survival models) to improve predictions and automate core processes. 2. MLOps & Model Lifecycle Management * Own the end-to-end ML lifecycle using Azure ML pipelines, Data Factory (ADF), and ML tracking tools. * Implement CI/CD pipelines with Azure DevOps, ensuring robust version control, testing, and seamless deployment. * Monitor production ML systems for performance, data … performance over time. 3. Data Engineering & Preprocessing * Develop and maintain scalable ETL/ELT data pipelines using Azure Data Factory, Data Lake, and SQL. * Prepare and preprocess large datasets to ensure data quality and readiness for ML modelling and analytics workflows. 4. Software Engineering & Collaboration * Write clean, maintainable, and More ❯

remote Benefits: Healthcare and great holidays The Role As a Machine Learning Operations Engineer, you will be instrumental in designing, deploying, and maintaining machine learning systems that drive data-driven decision-making in the pensions and financial services sector. Using Azure's cloud ecosystem, you will build scalable ML workflows that support actuarial modelling, risk prediction, and client … regression, ensemble methods, survival models) to improve predictions and automate core processes. 2. MLOps & Model Lifecycle Management * Own the end-to-end ML lifecycle using Azure ML pipelines, Data Factory (ADF), and ML tracking tools.* Implement CI/CD pipelines with Azure DevOps, ensuring robust version control, testing, and seamless deployment.* Monitor production ML systems for performance, data … performance over time. 3. Data Engineering & Preprocessing * Develop and maintain scalable ETL/ELT data pipelines using Azure Data Factory, Data Lake, and SQL.* Prepare and preprocess large datasets to ensure data quality and readiness for ML modelling and analytics workflows. 4. Software Engineering & Collaboration * Write clean, maintainable, and More ❯

Head of Data Engineering | High-Growth InsurTech London - Hybrid (Once per week in office)Up to £120,000 (Maybe £130,000) + Bonus + Share Options Are you a data leader who loves building high-performing teams and scalable platforms? I'm hiring for a Head of Data Engineering at a rapidly growing InsurTech … insurance works-making it simpler, fairer, and more personalised. This is a hands-on leadership role where you'll set the strategy, guide architecture, and empower a talented data engineering team to deliver world-class data systems in a fast-paced, product-led environment. Why This Role Stands Out Competitive salary + company share options Hybrid … ideas into business impact Passion for developing talent and fostering a culture of learning and ownership Nice to Have Experience with Data Mesh or Data Lake architectures Familiarity with Kubernetes , Docker, and real-time streaming (Kafka, Kinesis) Exposure to ML engineering pipelines or MLOps Interested? If you're ready to lead a team at the More ❯

Data Solutions Architect £90000-115000 GBP Bonus Hybrid WORKING Location: Central London, Greater London - United Kingdom Type: Permanent Data Solution Architect £90,000 - £115,000 + Bonus London - Hybrid - Permanent A global leader in IT Solutions and Consulting is seeking a Lead Data Solution Architect to join their dynamic, remote-first architectural team. You … will lead the design and delivery of innovative cloud-based data solutions for enterprise clients across various sectors. Working alongside talented Data Engineers, Business Analysts, and QA Analysts, you'll drive architectural best practices and deliver high-quality data solutions. This is an opportunity to own technical delivery, influence client architecture, and work with … Architect , leading end-to-end delivery of complex data solutions. Extensive expertise in data architecture design , including event-driven architectures (Kafka) and Data Lake/Lakehouse platforms . Hands-on experience with cloud platforms like AWS, Azure, GCP, or Snowflake. Strong knowledge of data governance, compliance, and security standards (GDPR, CCPA More ❯

Data Specialist - Government Client Are you a seasoned data professional ready to shape the future of data architecture in a high-impact government setting? We're working with a major government organisation seeking Data Specialists for upcoming projects. This is a strategic role focused on designing and delivering robust, interoperable data … or Manchester Security Clearance : You would have to undergo DV clearance (can take ~12 months; you may work other roles during this time) Key Responsibilities: Set and drive data architecture strategy across the organisation Promote and implement data standards and patterns with internal and external partners Ensure data interoperability and quality across delivery teams … Support cloud migration with a modern, data-centric architecture Useful Experience: Data Modelling : Physical, logical, conceptual models; data flow diagrams; ontologies; UML/Visio/Sparx Data Standards : Technical specs, code assurance, championing interoperability Metadata Management : Data catalogues, repositories; tools like Apache Atlas, Hive Metastore, AWS Glue/Datazone Data More ❯

In details, the position encompasses duties and responsibilities as follows: An experienced Data Engineer is required for the Surveillance IT team … to develop ingestion pipelines and frameworks across the application portfolio, supporting Trade Surveillance analysts with strategy and decision-making. The team has recently developed a new data lake house infrastructure and ingestion architecture and is entering a phase of hardening and re-engineering to enhance the speed, ease, and quality of ingestion across the portfolio. The ideal … of: Bachelor's degree in Information Systems, Computer Science, Engineering, or related field, or the equivalent combination of education, training and experience. 5+ years proven experience in a data engineering role demonstrating a strong track record of designing, building, and maintaining data pipelines and data architectures. Good communicator, with good interpersonal skills and the More ❯

talented and experienced Python Developer to join our dynamic team in London . You will play a critical role in developing and optimizing backend systems, working with modern data technologies, and contributing to high-performance and AI-enabled applications. Key Responsibilities: Develop robust, scalable applications using Python , APIs, and async processing techniques Collaborate with … cross-functional teams on system architecture, design, and implementation Work with Databricks or Microsoft Fabric to manage big data workflows Utilize Azure Cloud services (Data Lake, Functions, App Services) for development and deployment Implement and maintain containerized solutions using Docker and CI/CD pipelines Apply MLOps practices using tools like MLFlow , Git-based versioning … development experience with strong knowledge of APIs, data structures, and asynchronous programming Experience with Databricks or Microsoft Fabric Proficiency in Azure cloud services including Data Lake, Functions, and App Services Strong understanding of Docker , CI/CD workflows, and automation pipelines Familiarity with MLOps tooling such as MLFlow, Git version control, and environment management Desirable More ❯

A small and fast-growing Power Platform Consultancy is seeking a Microsoft Fabric Data Engineer to join their dynamic team in London, and play a key role in expanding their Microsoft Fabric offering. This is a fantastic opportunity to join a collaborative and forward-thinking consultancy that is passionate about delivering cutting-edge data solutions. You … ll be working closely with clients to help them unlock the full potential of their data using Microsoft Fabric, acting as a trusted advisor and technical expert. This role is ideal for someone currently working in a consultancy or someone eager to step into their first consulting role. In line with a hybrid working model, you'll spend … you should have: Proven experience working with Microsoft Fabric or strong expertise in related technologies such as Power BI, Azure Synapse, Data Factory, Azure Data Lake etc. A solid understanding of data engineering principles, including data modelling, ETL/ELT processes, and data warehousing. Hands-on experience with Power More ❯

About Moonbug Entertainment Thank you for considering the Data Lead role with Moonbug Entertainment, an award-winning global entertainment company inspiring kids everywhere to laugh, learn and grow. The company is behind some of the biggest kids' entertainment brands in the world including CoComelon and Blippi. Moonbug believes every child should have access to our entertaining and enriching … Benefits including Free Private Healthcare, Enhanced Maternity and Paternity leave, matched 5% pension scheme, free yoga/fitness wellbeing classes, free weekly lunch and Friday The Role The Data Lead is responsible for driving Moonbug's data engineering, platform, and governance strategy - ensuring our data infrastructure, pipelines, and practices are robust, scalable, and aligned … the CIO, works alongside the Tech Lead, and closely partners with the ML Lead, DataOps/Governance, and the BI & Analytics function to deliver a unified, high-quality data ecosystem. Key Responsibilities Lead data engineering & platform initiatives across onshore & offshore teams (Data Engineers, DB Admin, API Integration). Enable ML & AI teams by ensuring More ❯

objections, collaborating with engineering to share insights and improve products. Maintain deep expertise in Analytics Portfolio: Microsoft Fabric (OneLake, DW, real-time intelligence, BI, Copilot), Azure Databricks, Purview Data Governance, and Azure Databases: SQL DB, Cosmos DB, PostgreSQL. Maintain and grow expertise in on-prem EDW (Teradata, Netezza, Exadata), Hadoop & BI solutions. Represent Microsoft through thought leadership in … from migration & modernization and creating new AI applications. Expertise in Azure Analytics (Fabric, Azure Databricks, Purview) and competitors (BigQuery, Redshift, Snowflake) in data warehouse, data lake, big data, analytics, real-time intelligence, and reporting with integrated Data Security & Governance. Proven ability to lead technical engagements (e.g., hackathons, PoCs, MVPs) that drive More ❯

with other product delivery teams. You will provide technical leadership across workstream discussions, ensuring seamless collaboration where technical direction is required across multiple teams building applications over our data platform. Working with Solution Architects and Developers, you will actively participate in brainstorming and solution design sessions to address the needs and technical challenges facing those teams. You will … to guide them through the software development lifecycle. Provide guidance to, and coach and mentor, the teams in modern web application software development practices, with specific focus on data engineering. Facilitate continuous product and process improvements, assuring appropriateness and feasibility of proposed solutions as well as the continuous delivery of production software which improves the way we do … Skills/Qualifications Essential Skills Strong working knowledge of Azure Databricks/Spark (using Python and associated frameworks) Privacera and SQL Strong working knowledge of Azure Data Lake and Blob storage Strong experience of building data reporting and visualisations using PowerBI and/or Tableau Experience in application development with proficiency in Python or JavaScript More ❯