1 to 25 of 59 Data Lake Jobs in the City of London

Enterprise Data Solution Architect is responsible for defining and implementing the organization’s enterprise-wide data strategy and architecture. This role ensures that data assets are structured, governed, integrated, and leveraged effectively across all cloud environments. The ideal candidate has deep experience designing modern … data architectures on AWS, Microsoft Azure, and/or Google Cloud Platform (GCP) , and can guide business and technology leaders in building scalable, secure, and high-performing data ecosystems. Key Responsibilities 1. Data Strategy & Architecture Develop and maintain the enterprise data architecture ...

Role – Technology Lead Technology – Snowflake, SQL Server, PostgreSQL, Data Warehousing, Data Lake, Medallion, Data Vault, Data Fabric, Semantic modelling Location – London, UK Job Description We are seeking an experienced Data Modeler who can analyse and understand the existing customer … data and analytics models on SQL Server, and design efficient data models for Snowflake. This individual will work closely with multiple Business Analysts to gather requirements and translate them into optimized data structures, ensuring scalability and compliance with industry standards. A strong understanding ...

design end-to-end Azure data platforms , including ingestion, storage, transformation, analytics, and visualization layers. Build solutions leveraging Azure Data Lake, Synapse Analytics, Data Factory, Databricks, Event Hubs, Stream Analytics, Power BI , and related services. Translate complex business requirements into scalable, cost-efficient … Platform Architect with a primary focus on Azure . Hands-on expertise with Azure Data Factory, Synapse Analytics, Data Lake Storage (Gen2), Databricks, Power BI, Event Hubs, and Functions . Strong background in data modeling, data warehousing, and real-time ...

developers (quants and strategists) in a unique hands-on opportunity within a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with … What We’re Looking For 5 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience in implementing ...

unique opportunity to remain hands-on in engineering while also leading a focused, high-impact team. You’ll be at the forefront of data and cloud-native engineering, working with modern technologies across AWS, On-premise kubernetes, Python, and data pipelines, while engaging directly with … What We’re Looking For 10 + years, hands-on experience in AWS data engineering technologies, including Glue, PySpark, Athena, Iceberg, Databricks, Lake Formation, and other standard data engineering tools. Strong experience engineering in a front-office/capital markets environment. Previous experience in implementing ...

analytics-ready formats. Optimise pipeline performance and ensure high availability of data services. Infrastructure & Architecture Architect and deploy scalable data lake solutions using Azure Data Lake Storage. Implement governance and security measures across the platform. Leverage Terraform or similar IaC tools … Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations and CI/CD pipelines for Databricks deployments. Monitoring & Operations Implement monitoring solutions using Azure ...

customer insights critical for business growth, ensure robust security and compliance, and leverage cutting-edge Azure technologies such as Synapse, Data Lake, Databricks, and AI/ML services. The architect will play a pivotal role in enabling data-driven decision-making and modernizing the client … Services, Azure Data Factory (ADF), Azure Databricks, Azure Synapse, Azure SQL DB/DW Data Platform Expertise: Data Lake, Data Warehousing, ETL/ELT pipelines Expertise on DevOps & Automation: Azure DevOps (CI/CD), YAML pipelines, PowerShell scripting, Terraform/Bicep ...

best practice in DAX, performance tuning, and report design. Lead the development of data architecture and standards across Azure Data Lake and Synapse environments. Contribute to the migration to Microsoft Fabric, shaping platform architecture and integration patterns. Implement and maintain data validation, monitoring … managing enterprise models, datasets, and governance processes. SQL: Expert-level proficiency in query design, optimisation, and data transformation. Azure Data Lake: Deep understanding of data structures, storage layers, and integration for analytics. Microsoft Fabric (Emerging): Contribute to the organisation’s transition and adoption ...

Data Engineer - £350PD - Remote Required Technical Skills Data Pipeline & ETL Design, build, and maintain robust ETL/ELT pipelines for structured and unstructured data Hands-on experience with AWS Glue and AWS Step Functions Implementation of data validation, data quality … production pipelines Experience with incremental data processing patterns (CDC, watermarking, upserts) AWS Data Services Amazon S3: data lake architectures, partitioning strategies, lifecycle policies DynamoDB: data modeling, secondary indexes, streams, and performance optimization Amazon Redshift: foundational querying, integrations, and performance considerations ...

description S.H.I.F.T Talent has partnered with an IT Consultancy looking for Data Engineers. As a Lead Data Engineer you'll play a pivotal role in helping public sector organisations become truly data-lead, by equipping them with robust data platforms. … also join a data team on its mission to get data knowledge and skills out of silos and embedded into delivery teams. You'll also help implement efficient data pipelines & storage. Key responsibilities Define, shape and perfect data strategies in central ...

granular financial datasets. You’ll play a central role in shaping large-scale tick-data architecture - building pipelines, improving data-lake structures, and refining storage formats to support demanding research and trading workflows... Key Responsibilities Architect, build, and refine high-throughput pipelines capable of processing … Develop resilient, well-structured Python/OOP solutions underpinning tick-data ingestion, normalisation, storage, and replay Design and evolve data-lake patterns using Parquet and related columnar formats; introduce Iceberg-style table modelling where beneficial Implement cloud-backed processing and storage patterns - leveraging services from ...

Data Engineering Manager – Azure & Databricks Location: Hybrid (London-based) Salary: Competitive + Benefits Type: Permanent (Open to contract) About the Company Our client is a fast-growing Microsoft Partner consultancy on a mission to help businesses unlock the full potential of their data. They are building out their … roles, working your way up from a data engineer. Strong hands-on expertise with Azure Data Services (Data Lake, Synapse, Data Factory) and Databricks - need to be comfortable getting into the weeds of the work and building stuff yourself as well. ...

doing: Design, build, and optimise scalable data pipelines and architecture Work hands-on with SQL, PySpark, Databricks, Azure, and Data Lake to deliver high-performance solutions Translate business needs into technical requirements and data-driven solutions Ensure best practice in data … Engineer or Senior Data Engineer with some team leadership responsibilities Strong technical expertise with PySpark, Databricks, Azure, and Data Lake Deep understanding of data architecture and ETL processes in cloud environments The ability to balance coding and technical delivery with people management ...

Data Engineer Job Type : Permanent Location: Hub Location (London/Reading/Birmingham/Leeds/Glasgow) Hours of work: Full time (35 hours) with 3 days per week in the office Salary: £50,000 - £55,0000 Consultant: Mags Rendle Excellent career opportunity! The Organisation TDRC are excited … Analytics: Working knowledge of data modelling, views, and performance tuning. SQL: Proficient in writing, optimising, and troubleshooting queries. Azure Data Lake: Understanding of data storage and structure for analytics use cases. Git/Azure DevOps: Familiar with version control and Agile delivery workflows. ...

About the Opportunity This is a unique opportunity for a Data Engineer to work at the intersection of cutting-edge technology and elite sports performance. You’ll join a collaborative, high-standard engineering team with a culture built around creativity, innovation, and excellence. In this role … scientists to enable end-to-end data workflows Own the architecture and maintenance of a GCP-based data lake/lakehouse environment Your Background 4+ years of industry experience in Data Engineering roles Advanced-level Python for data applications ...

Permanent Opportunity for an eDV Cleared Data Engineer with AWS Cloud Experience for a leading National Security Consultancy in London. Up to £85,000 DoE plus bonuses Active eDV required London Location – full-time on-site when required Expertise required in AWS, Data Pipelines, ETL, Data … analytics and reporting. Develop ETL/ELT workflows to move data efficiently into data warehouses, data lakes and lake houses using open-source and AWS tooling. Apply DevOps practices, including CI/CD, infrastructure as code and automation, to improve and streamline data ...

Contract Data Architect Location: London (Hybrid) Contract Length: 12 Months Rate: £800 per day (Outside IR35) Senior Data Architect (Enterprise Data & Analytics) La Fosse are supporting a large, complex organisation undergoing a major business separation and transformation programme, and are looking for a Senior … data technologies, such as Oracle Fusion (Finance/HR/EPM), Microsoft Azure data platforms (Fabric, Synapse, Data Lake, Power BI), and enterprise integration tools (e.g. Azure Data Factory, Oracle Integration Cloud or equivalents). Proven involvement in carve-outs, demergers ...

Inside IR35 Roles & Responsibilities: We are seeking a highly experienced Senior Technical BA & Architect to lead enterprise-scale digital transformation programs across cloud, data, and application platforms. What You’ll Do Lead business and technical analysis across complex enterprise programmes Translate business needs into BDD, SDD, functional … security, and enterprise architecture teams Technology Stack Cloud: AWS (EC2, S3, RDS, Lambda, EKS, IAM, Neptune), Azure (ADF, Databricks, Synapse, Data Lake, AKS, Event Hub, Power BI) Architecture: Microservices, APIs, Java/Spring Boot DevOps: GitLab, Azure DevOps, Kubernetes, Helm, Terraform Data: Data ...

seeking an experienced Head of Data to join a global asset manager headquartered in London and lead the next phase of their data transformation journey. This is a strategic yet hands-on leadership role, suited to someone who excels at the intersection of data operations, governance, architecture, and emerging AI capabilities. Key Responsibilities Partner closely with senior business stakeholders to ensure the data strategy fully supports and enables business objectives. Work alongside the Head of Development to shape data architecture, modernise data tooling, and drive best ...

Senior Data Engineer £75,000 – £100,000 + bonus London (1–2 days per fortnight in office) Troi Search has partnered with a mission-driven healthtech company as they build out a new engineering pod. The organisation focuses on improving clinical workflows and driving better patient outcomes through … connected systems, data, and intelligent digital tools. Its platform is used by healthcare providers globally and continues to expand rapidly. Position Overview As the company grows, it is seeking a Senior Data Engineer to play a key role in developing and scaling its data ...

Data Engineer £45000 London (Once a week) Xcede is partnered with a mission driven organisation that is investing heavily in its data function. They are building modern data capabilities to improve insight, strengthen operations and support better decision making across the organisation. They … mart development • Contributing to platform security, risk awareness and disaster recovery activity • Helping to evolve and manage the organisation’s data lake • Promoting inclusive and values led ways of working You will bring • Experience delivering data engineering projects • Proficiency in SQL and Python • Exposure ...

Architect for London location JD Below ✅ Key Skills Technical Expertise Deep understanding of Azure Data Platform capabilities (Data Lake, Synapse Analytics, Data Factory, Databricks, Power BI). Knowledge of data architecture principles , data modeling, ETL/… processes, and cloud-native solutions. Familiarity with data governance, security, and compliance in Azure environments. Project Management Proven ability to manage end-to-end technical projects with budgets, timelines, and stakeholder alignment. Expertise in Agile and Waterfall methodologies , sprint planning, and resource allocation. Strong risk management , issue resolution ...

Role : Senior Data Architect Domain: Commodities Trading Location : London (hybrid 3 days per week) Contract : Initial 12 months Rate : £750-850/d inside IR35 X4 Technology are partnered with a global energy trading firm embarking on a major data transformation to enable AI-driven capabilities … confidently consumed by AI and analytics solutions by driving data productisation, enterprise & cross-portfolio modelling, business rule definition, Azure data lake maturity, data quality assurance and advanced concepts like ontology and data mesh. Your work will support large-scale trading ...

Senior Data Engineer, Azure London/hybrid Circa £90k-£100k + bonus + benefits Senior Data Engineer required by global banking organisation to help build and scale a modern data platform within a complex financial services environment. This role plays a key part … with technologies such as Azure Data Factory, Synapse, SQL Server and Power BI Solid understanding of data warehousing, lakehouse and datalake architectures Familiarity with modern data engineering patterns (ETL/ELT, medallion architecture, observability) Experience working with Apache Spark and large-scale data ...

functions. Gain access to endless learning opportunities. The Role As a MLOps Tech Lead, you will be responsible for designing, building, and maintaining data pipelines and infrastructure that enable the development and deployment of machine learning models and drive engineering excellence and collaborate closely with data scientists, and lead ML engineers, and software engineers to ensure data is clean, accessible, and optimised for large-scale processing and analysis. Your responsibilities: Data Pipeline Development: Lead the technical direction of projects and ensure the use of customer best practices to the best ...