1 to 25 of 49 Data Lake Jobs in London

more Chinese jobs please follow us on WeChat: teamchinapf AND pfteamchina Ref: 23377 The Skills You'll Need: Mandarin and English fluent, solid data management experiences within banking. Your New Salary: Competitive + Bonus + Benefits Fully office based Permanent Office based: London Start: ASAP Reports to: Head … report automation and process management tools Should be able to read and write SQL Should understand the architecture and concept of data lake and data warehouse related knowledge and design Must be proactive and analytical to evaluate and solve problems independently Must be able ...

IMMEDIATE JOINERS !! 💻 Junior Big Data Developer (Python & SQL Focus) 📊 We're looking for an enthusiastic and detail-oriented Junior Big Data Developer to join our data engineering team. This role is ideal for an early-career professional with foundational knowledge in data … distributed computing framework like Apache Spark (using PySpark). Familiarity with cloud data services ( AWS S3/Redshift, Azure Data Lake/Synapse, or Google BigQuery/Cloud Storage ). Exposure to workflow orchestration tools ( Apache Airflow, Prefect, or Dagster ). Bachelor's degree ...

data products, working closely with architecture where required. Implement and maintain data pipelines and ETL workflows in Databricks (PySpark, Delta Lake). Contribute to CI/CD pipelines for data applications using Azure DevOps and infrastructure-as-code (Terraform) in line with established … communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous ...

data products, working closely with architecture where required. Implement and maintain data pipelines and ETL workflows in Databricks (PySpark, Delta Lake). Contribute to CI/CD pipelines for data applications using Azure DevOps and infrastructure-as-code (Terraform) in line with established … communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous ...

Responsibilities Define the architecture and blueprint for a federated data platform built on Databricks Design and shape a centralised data lake architecture Translate business and PE investment metrics into measurable data engineering KPIs Identify data gaps, data availability issues … Architecture roles Strong experience with Databricks Deep understanding of data transformation, modelling, and engineering logic Experience designing data lake/lakehouse architectures Ability to translate business objectives into data architecture and KPIs Experience building data platform roadmaps and delivery ...

This is a rare opportunity to apply serious data engineering in a domain where latency, correctness, and reliability carry direct commercial weight. * Requirements * 6+ years data engineering in production environments; Python expertise - idiomatic, well-tested, production-grade code, not notebook scripts * ETL/ELT pipeline design … retry, and exactly-once semantics * Azure data platform - Azure Data Factory, Azure Databricks, Azure Synapse Analytics, Azure Data Lake Storage; infrastructure as code for data workloads (Terraform or Bicep) * Databricks - Delta Lake, Unity Catalog, job cluster vs interactive cluster trade ...

distribution keys, sort keys , and workload management. Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations. Build Iceberg-based ingestion and transformation pipelines using Glue …/Iceberg for low-cost, serverless analytics. Manage Glue Data Catalog integrations and table schema management. 5. Storage (S3) & Data Lake Architecture Design secure, cost-efficient S3 data lake structures (bronze/silver/gold zones). Implement data lifecycle ...

client is an international bank seeking a Data Management Officer to join its London-based IT function, supporting critical data governance and data security initiatives across the business. This is a hands-on role operating at the intersection of data, regulation … technology, working closely with Finance, Compliance, and broader business teams to embed robust data management practices across the organisation. Key Responsibilities * Support the implementation of group-wide data governance frameworks and standards * Assist in building and enhancing data quality frameworks, monitoring, and reporting mechanisms ...

Responsibilities Design, build, and maintain scalable data pipelines (primarily in Python) Architect and evolve a modern data platform (ingestion, lake, warehouse, and consumption layers) Develop and optimise data warehousing solutions, including ClickHouse and cloud-based platforms Implement and manage data orchestration … practices Experience with semantic layers and business-facing data models Track record of building end-to-end data platforms (ingestion → lake → warehouse) Comfortable working in an agile, iterative delivery environment The Offer Salary: Up to £110,000 Bonus: Strong performance-based bonus Location: London (Hybrid ...

critical functions. Responsibilities: Design, build and deploy scalable, secure data solutions using Azure Databricks, Data Factory and Data Lake Storage. Develop and optimise advanced data pipelines with Python, SQL, Spark/PySpark and Delta Lake. Champion strong data quality … business teams to align engineering solutions with organisational needs. Minimum Criteria Extensive experience with Azure services including Azure Databricks, Azure Data Lake Storage, and Azure Data Factory. Advanced proficiency in SQL, Python, and Spark (PySpark), with a strong focus on performance optimization and distributed processing. ...

sources Design and manage Purview scans, collections, and data maps for structured platforms including Azure SQL, Synapse, and Azure Data Lake Configure classifications, sensitivity labels, business metadata, and lineage for structured datasets Support the definition and implementation of structured data governance operating models … data governance use cases Proven experience working with structured data platforms such as Azure SQL, Azure Synapse, Data Lake Gen2, or similar Solid understanding of structured data concepts including relational models, data warehouses, and analytics architectures Experience designing ...

Office.Full time hours. We are happy to discuss flexible working! Top 3 skills needed for this role: Expert in building scalable, high-quality data pipelines Hands-on experience with cloud data platforms The ability to optimise, automate, and deliver reliable data solutions independently What … this role is all about: Join us as a Data Engineer and become a driving force behind high-impact, high-performance data solutions. Youll take ownership of designing and supercharging data pipelines that keep our business moving, bringing fresh thinking and technical flair ...

full of opportunities to turn today’s challenges into tomorrow’s solutions. Job Description Becoming part of the UK team in our Data & AI Service Line will present new opportunities for you to make a positive impact as we embark on securing major projects with our top international … working with our FTSE100 clients and beyond on a wide range of advisory projects, bringing deep business and technology expertise to help address Data and AI opportunities and challenges across their organisations. Wavestone has a significant and growing capability in Data & AI that is in high ...

Full time hours. We are happy to discuss flexible working! Top 3 skills needed for this role: Expert in building scalable, high-quality data pipelines Hands-on experience with cloud data platforms The ability to optimise, automate, and deliver reliable data solutions independently What … this role is all about: Join us as a Data Engineer and become a driving force behind high-impact, high-performance data solutions. You’ll take ownership of designing and supercharging data pipelines that keep our business moving, bringing fresh thinking and technical flair ...

Data Bricks Product Owner, Microsoft Fabric Product Owner, Data Engineering, AI Data Analytics, Microsoft Stack, Azure, Data Bricks, ML, Azure, Mainly Remote Data Bricks Product Owner, Microsoft Fabric Project Owner, Data Engineering/Technology Product Owner required … Engineering (Data Bricks, Microsoft Fabric etc) before moving into Product Ownership Technical proficiency:Familiarity with Azure services (e.g., Data Lake, Synapse, Fabric) and Databricks for data engineering, analytics, performance optimisation, and governance Development Framework experience within Microsoft Stack Technologies Experience with implementing ...

looking for a Data Engineer to join a fast-growing UK startup that’s reshaping how the hospitality industry manages staffing and operations through a data‐driven platform. You’ll play a key role in building the data foundations that power both internal decision … making and customer‐facing products. The role You’ll combine data engineering, analytics, visualisation and infrastructure to make high‐quality data accessible across the business. You will: Build and maintain scalable data pipelines feeding mobile and web apps, plus internal dashboards. Design and implement ...

Power BI dashboards and data models built on Charge Detail Record (CDR) data from the company's data lake Create new reports and visuals to track operational KPIs such as revenue per charger, utilisation rates, and site performance Work with the Microsoft Fabric … systems AAT qualification, part-qualified ACCA/CIMA, or relevant degree Experience working with API-sourced data or data lake environments What you'll get in return Hybrid working The opportunity to shape a growing finance function in a fast-scaling business Exposure to both ...

Data Engineer Contract Duration: 4 months Location: Farringdon, London Job Type: Contract, Hybrid (3 days a week onsite) We are seeking a Data Engineer to our client's ambitious technology transformation. This role is crucial in implementing the end-to-end delivery of data strategies, working closely with Data Analytics and Data Science teams to ensure best practices and standards are met. Day-to-day of the role: Source, map, and extract data from various business and cloud-based source systems. Design, develop, test, optimise, scale ...

Overview: We are working with a leading UK-based wealth and investment management firm seeking a skilled Data Engineer to support the build and evolution of their modern data platform. This role will focus on developing scalable data pipelines and contributing to a cloud … performance best practices are followed Required Skills & Experience: Strong experience with Azure data platform (e.g., Data Factory, Data Lake, Synapse) Hands-on expertise with Databricks (PySpark or Spark) Solid SQL development skills Experience working with Kafka for streaming data pipelines Good ...

engineers to deliver robust and reusable data products Manage data lakes and warehouses using formats such as Delta Lake and Parquet Implement best practices for data governance, performance, and security Continuously evaluate and adopt new technologies to evolve the organisation's data … junior engineers and contribute to team capability building Technical Stack Core: Azure Data Factory Azure Synapse Analytics Azure Data Lake Storage Gen2 SQL Server Databricks Enhancements: Python (PySpark, Pandas) CI/CD (Azure DevOps) Infrastructure as Code (Terraform, Bicep) REST APIs GitHub Actions Desirable: Microsoft ...

Permanent Opportunity for an eDV Cleared Data Engineer with AWS Cloud Experience for a leading National Security Consultancy in London. Up to £85,000 DoE plus bonuses Active eDV required London Location – full-time on-site when required Expertise required in AWS, Data Pipelines, ETL, Data … analytics and reporting. Develop ETL/ELT workflows to move data efficiently into data warehouses, data lakes and lake houses using open-source and AWS tooling. Apply DevOps practices, including CI/CD, infrastructure as code and automation, to improve and streamline data ...

Aker Systems was founded in 2017 by a team of experienced technology professionals who recognised an opportunity to provide highly secure enterprise data platforms to large organisations. Aker builds and operates ground-breaking, ultra-secure, high performance, cloud-based data infrastructure for the enterprise. Our proprietary … Platforms designing modern data platforms with appropriate architecture coving Data Mesh, Data Fabric, Data Lake etc. Data/Analytics - solving enterprise data and analytics problems using technology Cloud - hands-on Public Cloud experience in either ...

Title: Data Engineer – Azure Data Platform Location : London (Hybrid) Our client is hiring a Data Engineer to help build and scale enterprise-grade data solutions on the Microsoft Azure platform. This role involves delivering reliable, high-performance data pipelines that … optimization of data workflows Requirements: Experience in data engineering and pipeline development Strong expertise in Azure (Synapse, Data Lake) Proficiency in Python, PySpark, and T-SQL Knowledge of data warehousing, CDC, and incremental processing Familiarity with modern data architectures ...

consultancy seeking a Microsoft BI Stack Developer to design, build, and support end-to-end business intelligence and analytics solutions using the Microsoft data and analytics ecosystem. This is initially a 3 month contract with potential to extend and will be a hybrid role based in London … data services such as: Azure SQL Database/SQL Server Azure Synapse Analytics Azure Data Factory Azure Data Lake Support ingestion, transformation, and orchestration of data pipelines (in collaboration with data engineers where applicable). Governance, Security and Performance ...

Method are excited to partner with a prestigious London-based Wealth Manager looking to hire a Data Engineer to join their talented team as they enjoy a sustained period of growth. This business are renowned for their outstanding reputation in the space, strong career development opportunities, and access … need to succeed? Must-have Solid Data Engineering background across architectures such as Data Warehouse, Data Lake, or Data Lakehouse Strong SQL/T-SQL skills and hands-on experience with a cloud data platform (Databricks or Snowflake) Experience ...