1 to 25 of 174 Lakehouse Jobs in London

implementing resilient data ingestion, orchestration, and provisioning capabilities for enterprise-scale workloads. Strong hands‐on experience with Databricks (AWS and/or Azure), including Lakehouse architecture, distributed processing, and production operations. Strong hands‐on experience with Apache Kafka and event-driven integration architectures. Proven expertise designing and implementing enterprise ...

Skills: Advanced knowledge of cloud data platforms: Azure, AWS, and Google Cloud Platform (GCP). Strong experience with data warehousing, data lakes, and lakehouse architectures. Hands‐on experience with SQL and at least one programming language such as Python, Scala, or Java. ETL/ELT design and data pipeline ...

experience with: Azure Data Lake Databricks Snowflake Delta Lake Azure Synapse Analytics/Microsoft Fabric Spark/PySpark Strong understanding of: Data Lake and Lakehouse architecture Hybrid cloud environments Enterprise integration patterns Modern data formats including Parquet, Avro, and ORC DevOps & Platform Engineering Experience implementing CI/CD pipelines ...

Essential Experience Strong experience delivering enterprise-scale data architecture. Expertise in conceptual, logical, and physical data modelling. Experience with Oracle ERP, SQL databases, and Lakehouse platforms such as Databricks. Knowledge of cloud data platforms including Azure, AWS, or Google Cloud. Strong understanding of data governance, data quality, and master ...

Azure DevOps, CI/CD & Release Management Technical Documentation Technical Azure Databricks, PySpark, Python & SQL Azure Data Factory Azure Functions Azure Data Lake/Lakehouse REST APIs & Third-Party System Integration Azure SQL Logic Apps Azure DevOps, Git & CI/CD Docker/Containers Data Quality, Monitoring & Logging Data ...

Skills: Advanced knowledge of cloud data platforms: Azure, AWS, and Google Cloud Platform (GCP). Strong experience with data warehousing, data lakes, and lakehouse architectures. Hands-on experience with SQL and at least one programming language such as Python, Scala, or Java. ETL/ELT design and data pipeline ...

solutions using Azure Data Factory and Microsoft Fabric. Develop high-performance data processing solutions using Python, PySpark, and SQL. Implement Delta Lake and Lakehouse architectures following industry best practices. Work with Azure Data Lake Storage (ADLS Gen2) to manage large-scale datasets. Optimise Spark workloads for performance, scalability ...

experience delivering data and analytics solutions within an IT environment . Strong Microsoft Azure experience, particularly: Azure Data Factory Databricks Azure Data Lake/Lakehouse Good understanding of data architecture and data flows . Practical SQL experience. Experience with Python and/or PySpark . Experience with CI/ ...

enterprise and solution architects to align data engineering solutions with wider technology strategy and architecture standards. Drive adoption of modern data architecture approaches, including lakehouse and warehouse patterns, data product thinking, and reusable engineering services. Mentor and support data engineers through technical coaching, knowledge sharing, and career development activities. ...

data platform, ensuring reliability, performance and scalability * Design enterprise scale data architectures on Databricks and cloud platforms such as Azure, AWS or GCP * Define lakehouse standards, data modelling approaches and governance frameworks * Lead design of batch and streaming platforms using Delta Lake, Spark and related technologies * Establish security, access ...

/CD tooling such as Azure DevOps or GitHub Actions.* **Architectural Patterns:** Depth across architectural patterns such as microservices, event-driven, API-led, lakehouse/medallion and domain-driven design, with good judgement on when to apply each.* **Data Platforms:** A strong track record architecting data-intensive solutions handling ...

Functions, Storage, Service Bus, DevOps). Strong understanding of data architecture and ETL/ELT pipelines . Proven experience setting up Microsoft Fabric , Data Lakehouse , or Medallion architecture . Demonstrated ability to troubleshoot complex distributed systems and identify root causes. Exposure to SRE principles (monitoring, incident management, resilience). ...

architectures for production systems, with evidence that your designs have shipped and run - not remained on paper. Deep expertise in data modelling, warehouse/lakehouse design, and data governance. Hands-on background in SQL and Python, and fluency with at least one major cloud provider (AWS or Azure ...

databases (Cassandra, MongoDB, Cosmos DB). - Familiarity with data modelling, query optimisation, and indexing strategies. Data Platforms: - Hands-on experience with data lakes and lakehouse architectures. - Familiarity with big data processing frameworks like Spark. Data Engineering Knowledge: - Proven experience building ETL/ELT pipelines and using orchestration tools (Airflow ...

critical enterprise distributed systems.**Deep Technical Breadth & Depth:*** Advanced data architecture experience using Databricks, Microsoft Fabric, and large-scale data processing (Data Mesh/Lakehouse patterns).* Strong background in data integration and relational platforms* Proven experience designing highly available Hybrid and Multi-Cloud architectures (Azure & AWS). ...

maintain scalable data pipelines using Databricks, Apache Spark and Delta Lake * Develop and optimise batch and streaming workflows for reliability and performance * Contribute to lakehouse architecture and medallion pattern implementation * Implement data quality checks, monitoring and observability across pipelines * Apply platform security, access control and governance standards * Support code ...

Databricks and Snowflake; exposure to Azure or GCP is beneficial A good understanding of modern data engineering patterns including ETL/ELT, data lakes, lakehouse, and warehousing architectures Expopsure to data governance frameworks, metadata management, and regulatory complianc It would be a bonus if you have: Familiarity with emerging ...

with stakeholders. Requirements: 5+ years of hands-on data engineering experience, ideally within a consulting or client-facing environment Hands-on experience with Databricks Lakehouse Platform Strong SQL skills and proficiency in Python Experience designing and deploying solutions in cloud environments Ability to quickly grasp diverse business domains ...

national security or another highly regulated environment. Knowledge of AWS, Azure, Microsoft Fabric or comparable cloud data platforms. Experience with data lakes, data warehouses, lakehouse architectures and real-time or event-driven data solutions. Understanding of APIs, microservices and modern systems-integration approaches. Familiarity with data cataloguing, master data ...

experience building real-time event streaming pipelines using technologies such as Kafka, Spark Streaming, or Amazon Kinesis within AWS environments.Proven expertise designing modern Data Lakehouse and Data Warehousing architectures, alongside a solid understanding of CI/CD, DevOps practices, and GDPR-compliant data governance.Experience delivering scalable cloud-based data ...

Senior Software Engineer on the Data Engineering team, you are a core builder of the Caspian Data Platform — the company's centralized lakehouse that powers analytics, financial reporting, product intelligence, and data-driven operations across every business unit. You own the design and delivery of production-grade pipelines ...

take ownership of getting good quickly. People who wait to be trained will find this uncomfortable.Nice to have: Experience building Data Mesh or Lakehouse architectures.Familiarity with Kubernetes, Docker, and real-time streaming technologies (e.g. Kafka, Kinesis).Exposure to ML engineering pipelines or MLOps frameworks.BenefitsWhat it's like to work ...

tests in a marketing context Desirable Experience with dbt (Cloud or Core) for transformation layer development Familiarity with Databricks or similar modern data lakehouse platforms Experience with Power BI (or similar data visualisation tool) Experience in a regulated financial services or fintech environment Exposure to customer data platforms (CDPs ...

tests in a marketing context Desirable Experience with dbt (Cloud or Core) for transformation layer development Familiarity with Databricks or similar modern data lakehouse platforms Experience with Power BI (or similar data visualisation tool) Experience in a regulated financial services or fintech environment Exposure to customer data platforms (CDPs ...

Experience building and designing products that use both real-time and batch data at scale, and an understanding of the underlying data architecture (e.g. lakehouse, star schema). Ability to write and validate SQL queries, and to trace data flows end-to-end to ensure data quality. Awareness ...