51 to 75 of 272 Lakehouse Jobs in London

workflow orchestrator such as Airflow, Dagster or Prefect, and transformation tooling such as dbt or equivalent SQL-based workflows Cloud warehouse, lake or lakehouse experience , and integrating data from external APIs, databases and files Correctness under failure. Understanding of schema design, data contracts, idempotency, replay, backfills and late-arriving ...

establishing engineering standards that improve reliability, maintainability, scalability and reuse.Your experience is likely to include several of the following: Microsoft Fabric Snowflake Azure OneLake Lakehouse architecture Data Factory Spark Python SQL ETL/ELT dimensional modelling Data Vault orchestration Git Azure DevOps/GitHub CI/CD Infrastructure … dependency management, exception handling and delivery. Design and assure robust analytical data models translate complex analytical and business requirements into well-engineered, reusable dimensional, Lakehouse, Data Vault and semantic models that improve analytical consistency and performance. Drive automation and analytical productivity identify manual, repetitive and inefficient processes within Data ...

models Design secure, scalable, resilient, and supportable data platforms and solutions Define patterns for data integration, transformation, storage, and analytics Support the design of lakehouse, warehouse, and reporting solutions using Azure-based services Ensure data quality, lineage, governance, and security are incorporated into solution design Collaborate with engineers … skills Self-sufficient and collaborative, with the ability to work independently, alongside a structured, detail-oriented, and outcome-focused approach Experience with Microsoft Fabric, lakehouse architectures, and hands‐on development using Python, SQL, or Spark would be advantageous, along with familiarity with integration technologies Knowledge of DevOps practices, Infrastructure ...

AI. Modernisation is at the heart of the role: helping customers move away from legacy data warehouses, databases and fragmented data estates towards modern lakehouse, warehouse and federated designs on platforms such as Databricks, Snowflake, Microsoft Fabric and Starburst. In a pre-sales capacity, there is a demand … define migration approaches and phased roadmaps, and identify risks, dependencies and quick wins. Data Architecture & Modelling - Design end-to-end data platform architectures, including lakehouse, warehouse, federated and data-product approaches. Apply appropriate modelling patterns such as medallion, dimensional and domain-oriented designs. Data Integration & Engineering - Design ingestion ...

driving best practices, and guiding engineering teams to deliver scalable, secure, and cost optimized data platforms. \* Design end to end data platforms leveraging Databricks Lakehouse architecture, Delta Lake, Unity Catalog, and ML/AI workloads. \* Define data ingestion, transformation, governance, and consumption frameworks aligned to enterprise standards. \* Evaluate ...

agents. Headquartered in San Francisco with 30+ offices around the globe, Databricks offers a unified platform that includes Genie, Lakebase, Agent Bricks, Lakeflow, Lakehouse, and Unity Catalog. To learn more, follow Databricks on LinkedIn , X , YouTube , and Instagram . At Databricks, we strive to provide comprehensive benefits and perks ...

data architect or senior data engineer working on large‐scale data and analytics platforms. Deep understanding of modern data architectures (e.g. data lakehouse, event‐driven architectures, streaming, data mesh principles). Strong knowledge of one or more leading cloud data stacks (e.g. GCP BigQuery, AWS Redshift/Glue, Azure ...

pipelines in AWS and/or GCP. Comfortable with data quality tooling and the mindset that correctness is non‐negotiable. Familiarity with medallion/lakehouse architecture patterns. Verbal and written fluency in English is required — we're a distributed team and written communication is important. Preferred Qualifications Airflow ...

advising on migration architecture, transition states, historical data migration and legacy decommissioning. Lead the architecture, design and development of scalable cloud data warehouse, lakehouse, streaming and analytics solutions. Define ingestion, transformation and orchestration patterns, naming and structural conventions, and the publication boundary between Product Engineering and Data Engineering. Identify ...

Data Modelling & Storage: Strong data modelling skills and the ability to select appropriate data stores—including relational, NoSQL, graph, time‐series, distributed SQL and lakehouse technologies—based on business and technical requirements. End‐to‐End Data Flows: Architect operational and analytical data pipelines across APIs, streaming, CDC, batch ...

strong track record of mentoring engineers, leading projects, and a clear desire to step into people management. Deep experience with a modern lakehouse, ideally Databricks and Unity Catalog, and with dbt. Comfortable on AWS and with infrastructure as code (Terraform), and able to hold your own in a design ...

track record in leading the architecture vision, design & implementation of Enterprise scale data applications, with specific focus on Distributed Data Processing systems, Data LakeHouse, Data Products, Streaming data, ML & AI platform capabilities* Working knowledge of ML and Generative AI, including the leverage of Agentic AI based systems within ...

Experience delivering AI solutions in areas such as risk, compliance, fraud, customer operations, lending, or regulatory reporting. Data architecture, data modernisation, data platforms, or lakehouse expertise. Experience supporting large-scale cloud migration or transformation programmes. Knowledge of AI governance, model risk management, and regulatory requirements within financial services. Exposure ...

with the Program Office for planning deliveries, providing timelines, cost estimates, and status updates. Partner with Data Engineering to ensure data pipelines and lakehouse structures support AI delivery needs. Team Development: Support the Head of Data Science & AI Delivery in leading and developing a growing team of Data Scientists ...

advising on migration architecture, transition states, historical data migration and legacy decommissioning. Lead the architecture, design and development of scalable cloud data warehouse, lakehouse, streaming and analytics solutions. Define ingestion, transformation and orchestration patterns, naming and structural conventions, and the publication boundary between Product Engineering and Data Engineering. Identify ...

support trusted data products and star schemas for actionable operational, tactical and strategic insights. These models will increasingly be implemented within a modern Azure lakehouse environment and consumed through Power BI and other governed analytics services. You will support these models with artefacts such as business glossary and data … entries. As L&Q evolves its data platform, the Data Modeller will contribute to the definition and adoption of modelling standards for a medallion lakehouse architecture, working with Data Engineering, Data Governance, Data Quality and Analytics colleagues. The role will help translate platform requirements into practical modelling patterns across ...

alongside the Data Programme to build the foundational data capabilities required for those products: trusted, reusable data assets, reliable pipelines, data quality, lakehouse models and analytical products. These capabilities will support R&D products, AI applications, client intelligence and firm-wide decision support. This is a hands-on hybrid … Microsoft Fabric and Databricks to ingest, clean, standardise, enrich, version and publish structured, semi-structured and unstructured data as reusable data assets. Create scalable lakehouse data models, schemas, data contracts and reusable features for entities such as clients, organisations, people, matters, sectors, jurisdictions, opportunities and legal topics. Implement data ...

accessible data drives real business outcomes and supports growth across regulated industries.ResponsibilitiesImplement end-to-end Microsoft Fabric solutions based on approved architecture, including OneLake, Lakehouse, Warehouse, Data Factory pipelines, Dataflows Gen2, Notebooks (PySpark/Spark SQL) and Real-Time IntelligenceBuild medallion architectures (Bronze/Silver/Gold), including incremental … notebooksHands-on experience with Azure data services (Azure Data Factory, ADLS Gen2, Azure SQL, Synapse or Databricks)Solid understanding of data warehousing and lakehouse concepts, Delta/Parquet formats and dimensional modeling (Kimball)Experience with Real-Time Intelligence (Eventstreams, Eventhouse/KQL, Activator)Ability to turn architecture documents ...

approaches. Provide technical leadership across modern data platforms, helping clients make appropriate architecture and technology decisions. Databricks & Modern Data Platforms Design enterprise Databricks and Lakehouse solutions incorporating technologies and patterns such as Delta Lake, medallion architecture, Unity Catalog and Databricks SQL. Apply strong knowledge of modern cloud data ecosystems … Include the Following? Significant experience designing and delivering enterprise cloud data architectures within complex client or business environments. Deep expertise in Databricks and modern Lakehouse architecture, including practical knowledge of Delta Lake and Unity Catalog. Strong experience with Microsoft Azure and/or AWS and modern enterprise data platforms ...

Strong communication and stakeholder engagement skills. Required Skills Power BI DAX Semantic Modelling Power Query Databricks SQL Data Visualisation Desirable Experience Azure Databricks/Lakehouse environments Power BI Service Deployment pipelines/DevOps Large‐scale enterprise reporting programmes Microsoft Fabric #J-18808-Ljbffr ...

Experience working with modern enterprise data technologies such as Snowflake and/or Databricks Good understanding of enterprise data platforms, data architecture, on-premise lakehouse architectures and enterprise systems Knowledge of DevOps, CI/CD, Infrastructure as Code and modern testing practices Agile delivery experience (Scrum/Kanban) Experience ...

technical, or operational objectives into manageable tasks and activities Preferred qualifications, capabilities, and skills Experience delivering on modern data stacks (e.g., cloud data platforms, lakehouse/warehouse patterns, streaming platforms) and partnering with architecture on scalability, resiliency, and cost-to-serve. Experience with governance and controls: data access controls ...

agents. Headquartered in San Francisco with 30+ offices around the globe, Databricks offers a unified platform that includes Genie, Lakebase, Agent Bricks, Lakeflow, Lakehouse, and Unity Catalog. To learn more, follow Databricks on LinkedIn , X , YouTube , and Instagram . Benefits At Databricks, we strive to provide comprehensive benefits ...

them to guide roadmap priorities and architecture governance decisions. PERSONAL PROFILE Deep expertise in Databricks (Unity Catalog, Delta Lake, Spark, Databricks SQL) or equivalent Lakehouse platforms, including administration, cost management, and performance optimisation. Strong understanding of cloud-native architecture principles, cost models, and practical experience with CI/ ...

govern, and activate customer data while maintaining full ownership and control. By running directly on top of the customer's cloud data warehouse or lakehouse, RudderStack reduces data movement, improves governance, lowers infrastructure costs, and accelerates modern data and AI initiatives. Leading companies including Lovable, Jaguar Land Rover ...