1 to 25 of 115 Data Lake Jobs in London

Acquisitions, and Citi Commercial Bank client segments in over 100 countries. As a critical component of the TWCS Business Execution team, the Data Organization is responsible for establishing and implementing the overall data strategy and governance framework, ensuring high-quality data to drive business … value. This Director-level role is a senior global position responsible for defining and executing the data strategy and roadmap, aligned with various Lines of Business (LOBs) that act as domain owners for their respective products. The role is pivotal in delivering the Technology Book of Work assigned ...

build tool) – modelling layers, testing, documentation, incremental strategies Application Databases Data Integration AWS – S3, RDS, Aurora, Redshift (migration context), Glue, Lake Formation, IAM, VPC Data Governance Data lineage, cataloguing (Apache Atlas/Collibra/Snowflake Horizon), GDPR, BCBS … contracts, schema versioning Redis Cache invalidation, persistence strategy, data consistency MongoDB TimescaleDB NATS JetStream AWS S3/Glue Data lake staging, archival, batch ingestion into Snowflake Partitioning, file format (Parquet/ORC), Lake Formation governance Finance Domain Knowledge Candidates should have hands ...

Acquisitions, and Citi Commercial Bank client segments in over 100 countries. As a critical component of the TWCS Business Execution team, the Data Organization is responsible for establishing and implementing the overall data strategy and governance framework, ensuring high‐quality data to drive business … value. This Director‐level role is senior and global, responsible for defining and executing the data strategy and roadmap in alignment with various Lines of Business (LOBs) that act as domain owners for their respective products. The role is pivotal in delivering the Technology Book of Work assigned ...

warehouse architectures: Implement logical and physical data models, data quality controls, and governance practices using Amazon S3 (data lake), AWS Lake Formation, Amazon Redshift, and Amazon Athena to ensure trusted data.* Ability to design AI/ML-enabled solutions: Enable feature-ready … governed delivery.* Stay ahead of the curve: Continuously learn and adapt to the evolving AWS data and analytics landscape, including AWS Glue, Lake Formation, Redshift, Athena, EMR, and related best practices.* Define the platform operating model: Establish AWS multi-account strategies, environment separation (dev/test/ ...

IaaS, specifically: App Services, Azure SQL Database, SQL Server, Storage, Synapse Analytics, Purview, Databricks, Azure data Factory, Azure data lake & Fabric. Understand which Azure data focused components to use for which purpose.Microsoft certified architectDatabricksDesign and build high-performance data platforms: Utilize … Databricks and Apache Spark to extract, transform, and load data into Azure Data Lake Storage and other Azure services.Ability to Design, Build and deploy AI/ML models: Integrate Machine Learning into data pipelines, leverage Databricks ML and Azure ML to develop predictive ...

member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. Job Title : Senior Data Modeler - Insurance Location : London About the Job you are considering: We are seeking an experienced Senior Data Modeler with deep expertise … Cloud Platforms Experience in one or more: Azure AWS GCP Data & Analytics Technologies Advanced SQL Data Warehousing Data Lake Architecture Metadata Management Data Governance Frameworks Mandatory Insurance Domain Expertise Policy Administration Policy Lifecycle Management Policy Issuance Renewals Endorsements Cancellations Claims Management ...

umbrella) Duration: Initial 6 months+ Robert Half are supporting a leading property and real estate organisation on a major Finance Data Lake and Data Transformation programme and are looking for an experienced Data Architect to join the team. This role requires a hands …/CI/CD experience within data environments Strong stakeholder engagement and solution design capabilities Experience delivering enterprise-scale data lake initiatives Desirable: Finance Data Lake programme experience Lead Data Engineering background Experience working within large transformation programmes Robert Half ...

initiative delivery outcomes. The supplier will validate data across the full data lifecycle, including ingestion, transformation, data lake, data warehouse, reporting and BI, APIs, metadata, lineage, and reconciliation.**Assignment Tasks/Deliverables**Full Stack Data Quality Engineering:* Design … execute data quality testing across ingestion, transformation, warehouse, reporting/BI, APIs, metadata, lineage, and reconciliation layers.* Validate Snowflake data lake and data warehouse outputs against source systems, business rules, transformation logic, and reporting requirements.* Test ETL and ELT processes, dbt models ...

every report and every operational data feed at ECP, continue the migration of our data warehouse and data lake to a modern dimensional model, finish the rollout of a principle-of-least-privilege security model across a multi-server SQL estate, and continue … personally delivering target architecture for the ECP data estate the data warehouse, data hub, data lake, raw/ODS layers, dimensional model, audit and schema-change tracking, and the load schedules that tie them together. Hands-on T-SQL, stored procedure ...

Description The Role We are seeking a capable Consulting Data Architect to join our Insurance Technology Consulting practice and play a pivotal role in shaping, selling, and delivering data-led consulting engagements for insurance clients. Location: London, United Kingdom (Hybrid) Scope: UK-Based Role with Global … secure enterprise data platforms using technologies such as Microsoft Fabric, Azure Synapse Analytics, Azure Data Factory, Azure Data Lake, Databricks, and Snowflake. Data Management, Governance & Controls – Experience establishing data governance frameworks, operating models, metadata management, lineage, cataloguing, data ...

Olympic Committee, and Mercedes-Benz. Our services span the full digital delivery lifecycle, including architecture, engineering, delivery management, user-centred design, business analysis, data, DevOps, and AI. We operate as a collaborative and inclusive organisation that empowers our people to take ownership, innovate, and develop their expertise. … approach, you’re comfortable engaging with clients, understanding their needs, and translating them into effective outcomes. The Role: We are looking for experienced Data Architects to work on our projects with our public sector clients, helping to deliver Solirius’ services to the highest standard. The role will involve ...

experience with Azure Cosmos DB, including data modelling, partitioning, indexing, throughput optimisation, and Change Feed processing. Experience with Azure Data Lake Storage (ADLS), OneLake, ABFS, and Azure Key Vault. Strong understanding of data modelling concepts, including temporal data patterns such … , including ISINs, CUSIPs, LEIs, PermIDs, and asset classifications. Experience with entity resolution, record matching platforms, and third-party enrichment services. Experience with Delta Lake and Change Data Feed (CDF) patterns. Knowledge of Cosmos DB SDK optimisation techniques, including bulk operations, optimistic concurrency, and RU optimisation. Experience ...

time data pipelines that ingest data from legacy and modern platforms Lead the development of cloud-based data lake architectures within AWS Build streaming data solutions using Kafka Design and implement Medallion Architecture (Bronze, Silver, Gold layers) to support analytics … Essential Experience Strong hands-on expertise with Databricks Extensive experience using Spark, PySpark and SQL Proven track record designing and implementing data lake architectures Experience building both batch and streaming data pipelines Strong knowledge of Kafka Good understanding of AWS data services, particularly ...

Senior Databricks Architect Technology: Databricks (Spark, PySpark/SQL) and data lake architectures. Location: London, UK. Compensation: Competitive (including bonus). Job Description We are looking for a Data Architect to design, build, and operate scalable data pipelines and data products … support analytics, reporting, and domain-owned data products. Primary Skills Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures. Hands‐on experience with streaming technologies, especially Kafka/Confluent Kafka. Experience integrating legacy data sources such as DB2 (mainframe ...

Role –Senior Databricks Architect Technology – Databricks (Spark, PySpark/SQL) and data lake architectures Location – London, UK Duration-Permanent Job Description We are looking for a Data Architect to design, build, and operate scalable data pipelines and data products, enabling … support analytics, reporting, and domain-owned data products. Primary Skills • Strong experience with Databricks (Spark, PySpark/SQL) and data lake architectures. • Hands-on experience with streaming technologies, especially Kafka/Confluent Kafka. • Experience integrating legacy data sources such as DB2 (mainframe ...

member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. Job Title : Cloud Data Engineer About the Job you are considering: We are seeking an experienced Cloud Data Engineer with expertise in AWS, Snowflake, dbt, PySpark … Manager Data Modelling Skills Star Schema Snowflake Schema Dimensional Modeling Data Vault Data Warehouse Design Data Lake Architecture Data Formats Experience working with: Parquet Avro JSON CSV ORC CI/CD & DevOps Git GitHub GitLab Bitbucket Jenkins GitHub Actions ...

optimise scalable ETL/ELT pipelines using modern cloud data integration and orchestration services Design and implement secure, scalable data lake, lakehouse, and analytics platforms supporting structured, semi-structured, and unstructured data Establish robust data governance, quality, lineage, metadata management … Azure Functions, Container Apps, Azure Kubernetes Service (AKS) Docker and Kubernetes Storage & Data Amazon S3, Azure Blob Storage, Azure Data Lake Storage Gen2 Amazon Aurora PostgreSQL, RDS, DynamoDB Azure Database for PostgreSQL, Azure SQL Database, Cosmos DB Integration & Messaging Amazon EventBridge, SQS, SNS, API Gateway ...

work. Join us and build for travelers everywhere.Expedia Group seeks a motivated and results-oriented Senior Software Engineer to join our Data Lake and Catalog team.As a Senior Software Engineer on an industry-leading data platform, you will build the core of Expedia Group … with tools like Circus Train , a Hive dataset replication tool, and Waggle Dance , a federation service that allows access to data lake tables across multiple catalogs. Recently, we added Hive support to Apache Iceberg , a high-performance open table format for large analytic datasets that underpins many ...

Title: HR Data Delivery Lead Role Overview A new data lake solution is being implemented and the HR Data Delivery Lead will be responsible for end-to-end delivery of data products, pipelines, and reporting solutions across multiple teams. This role … will be accountable for connecting data from the new Oracle HCM Cloud to the new AWS Snowflake solution and unifying data layer between legacy and new HRIS system feed to data warehouse. The role ensures high-quality, timely, and governed delivery of data ...

Architect - London (Hybrid) - Databricks - Spark - PySpark - AWS - Up to £115,000 + Bonus Our client, a large-scale organisation undergoing significant investment in data and cloud technologies, is seeking an experienced Senior Databricks Architect to join its expanding Data & AI practice. This is an exciting opportunity … Data Engineer or Solutions Architect Strong expertise in Databricks, Apache Spark, PySpark and SQL Experience designing and implementing modern data lake or lakehouse architectures Hands-on experience with Kafka Strong understanding of AWS data services, particularly Amazon S3 Experience integrating data ...

Senior Data Engineer Market Data 5 day onsite We are looking for a Senior Data Engineer to build and operate a high-performance market data platform supporting quantitative research and production trading systems. You'll design scalable data pipelines, process … actions. Implement automated data quality, reconciliation and monitoring frameworks. Optimise data storage using Parquet, Arrow and modern data lake technologies. Partner with quantitative researchers, traders and engineering teams to deliver trusted datasets. Essential Skills Strong Python and SQL. Experience building scalable, production data ...

controls and operational guardrails Technical Leadership & Standards Serve as the senior technical authority for the Databricks platform architecture Establish engineering best practices covering Delta Lake design, workload orchestration, compute optimization, and cost management Provide technical assurance across platform implementation activities delivered by internal teams and external partners Define reusable … stakeholder engagement, and consulting capabilities Experience translating business requirements into strategic platform architecture solutions Technical Skills Deep expertise in Databricks, including Lakehouse Architecture, Delta Lake, Unity Catalog, Workflows, and cluster management Strong experience designing cloud-native data platforms on AWS Hands‐on knowledge of AWS services including ...

considering: We are looking for an experienced AWS Databricks Engineer with strong hands-on expertise in Databricks, AWS cloud services, PySpark, Spark SQL, Delta Lake, Python, SQL, and data engineering. The candidate will be responsible for designing, developing, optimizing, and supporting scalable data pipelines … maintain scalable data pipelines using Databricks on AWS. Build and optimize data processing frameworks using PySpark, Spark SQL, Python, Delta Lake, and Databricks notebooks. Develop end-to-end ETL/ELT pipelines for ingestion, transformation, validation, and consumption. Work with AWS services such ...

Principal Data Engineer Lead the design and delivery of a modern cloud-based data platform, driving engineering standards, architecture, and innovation across a growing data function. The Opportunity This is a senior technical leadership role responsible for shaping data engineering strategy, architecture … best practices across multiple product domains. Working within a modern cloud environment and Data Mesh architecture, you will drive the development of scalable, secure, and high-performing data solutions while mentoring engineers and influencing technical direction. Key Responsibilities Provide technical leadership across data engineering ...

Products & Insights group. You will manage a small team of product managers, including the Product Manager who owns our new data lake, and keep hands-on product work of your own: the foundations that let teams across Wise build with GenAI in production. What … leadership and partner teams, and bring company context back to the team Guide the Product Manager's who owns our data lake and analytics platforms, helping them build product craft and stay close to the teams who depend on them. Stay accountable for the group's delivery ...