1 to 25 of 89 Delta Lake Jobs in London

intersection of consulting, technology, innovation, and strategic alliances and you will bring deep expertise in the Databricks Data Intelligence Platform, Lakehouse architecture, Delta Lake, Apache Spark, Unity Catalog, MLflow, Mosaic AI, data engineering, machine learning, GenAI and cloud-native delivery. As a key leader within our AI & Data … value pools, translate business needs into Databricks-led solution designs, build prototypes and proof points. Shape end-to-end solutions using Databricks Intelligence Platform, Delta Lake, Apache Spark, Unity Catalog, Databricks Workflows, Delta Live Tables/Lakeflow-style pipelines, MLflow, Feature Store, Model Serving and Mosaic ...

intersection of consulting, technology, innovation, and strategic alliances and you will bring deep expertise in the Databricks Data Intelligence Platform, Lakehouse architecture, Delta Lake, Apache Spark, Unity Catalog, MLflow, Mosaic AI, data engineering, machine learning, GenAI and cloud-native delivery.As a key leader within our AI & Data Practice … value pools, translate business needs into Databricks-led solution designs, build prototypes and proof points.• Shape end-to-end solutions using Databricks Intelligence Platform, Delta Lake, Apache Spark, Unity Catalog, Databricks Workflows, Delta Live Tables/Lakeflow-style pipelines, MLflow, Feature Store, Model Serving and Mosaic ...

intersection of consulting, technology, innovation, and strategic alliances and you will bring deep expertise in the Databricks Data Intelligence Platform, Lakehouse architecture, Delta Lake, Apache Spark, Unity Catalog, MLflow, Mosaic AI, data engineering, machine learning, GenAI and cloud-native delivery. As a key leader within our AI & Data … value pools, translate business needs into Databricks-led solution designs, build prototypes and proof points. • Shape end-to-end solutions using Databricks Intelligence Platform, Delta Lake, Apache Spark, Unity Catalog, Databricks Workflows, Delta Live Tables/Lakeflow-style pipelines, MLflow, Feature Store, Model Serving and Mosaic ...

platforms such as Azure, AWS or GCP * Define lakehouse standards, data modelling approaches and governance frameworks * Lead design of batch and streaming platforms using Delta Lake, Spark and related technologies * Establish security, access control, data quality and compliance standards * Optimise performance, scalability and cloud cost management * Lead … experience * Strong experience as a senior or lead data engineer building and scaling production data platforms * Deep hands on expertise in Databricks including Spark, Delta Lake and Databricks Workflows * Python and SQL proficiency * Infrastructure as code experience using Terraform, Bicep or CloudFormation * Cloud data platform experience, ideally Azure ...

Data Engineer and partnering with data science, analytics and product teams. Key responsibilities * Build and maintain scalable data pipelines using Databricks, Apache Spark and Delta Lake * Develop and optimise batch and streaming workflows for reliability and performance * Contribute to lakehouse architecture and medallion pattern implementation * Implement data quality … opportunities * Translate business requirements into technical solutions * Contribute to documentation and continuous improvement Required experience * Strong experience delivering production grade Databricks pipelines * Strong Spark, Delta Lake and Databricks Workflows experience * Python and SQL proficiency * Experience on Azure data platforms such as Azure Data Factory, ADLS Gen2, Azure DevOps ...

implement scalable Databricks Lakehouse platforms on AWS and/or Azure aligned to client requirements Architect end-to-end data platforms including ingestion, storage (Delta Lake), processing, and consumption layers Build and configure cloud infrastructure using infrastructure-as-code (e.g. Terraform & Declarative Automation Bundles(DAB's)) Establish secure … pipelines, and release processes for Databricks deployments Implement monitoring, logging, cost optimisation, and performance tuning across the platform Design and implement data pipelines using Delta Live Tables, Auto Loader, and Databricks Workflows for both batch and streaming workloads Client Delivery & Enablement responsibilities: Work directly with clients to translate business ...

Senior Data Engineer (Databricks) - London (Hybrid) - Up to £85,000 Databricks - Apache Spark - Delta Lake - Python - SQL - Azure We're partnering with a fast-growing technology business that's investing heavily in its data platform and engineering capability. This is an opportunity to join a highly collaborative team … growing technology business. Key Experience Required Proven experience as a Data Engineer within production environments. Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python, and SQL. Experience building and supporting scalable ETL/ELT pipelines. Strong understanding of distributed data processing and data modelling. Cloud platform ...

practices.What You'll DoArchitect and lead the end-to-end delivery of Databricks-based data platforms across Azure environments.Design secure, scalable data pipelines using Delta Lake, Spark Structured Streaming, Unity Catalog, and Azure-native services, applying AI-enabled engineering approaches where appropriate.Establish and promote engineering best practices across … approved AI tools and automation to enhance delivery and collaboration.What We're Looking ForProven expertise with the Databricks platform, including Unity Catalog, Delta Lake, orchestration, and enterprise-scale Azure deployments.Strong technical skills in Python, PySpark, and distributed data processing frameworks, with experience building production-grade data platforms.Extensive experience ...

Lead Data Engineer (Databricks) - London (Hybrid) - Up to £110,000 Databricks - Apache Spark - Delta Lake - Python - SQL - Azure We're partnering with a growing technology business investing heavily in its cloud data platform and AI capabilities. This is a senior, hands-on leadership role where … machine learning and large-scale data initiatives. Key Experience Required Proven experience as a Senior or Lead Data Engineer. Deep expertise in Databricks, Spark, Delta Lake, Python and SQL. Experience designing and scaling cloud-based data platforms. Strong knowledge of ETL/ELT, data modelling and distributed processing. ...

practices, and guiding engineering teams to deliver scalable, secure, and cost optimized data platforms. \* Design end to end data platforms leveraging Databricks Lakehouse architecture, Delta Lake, Unity Catalog, and ML/AI workloads. \* Define data ingestion, transformation, governance, and consumption frameworks aligned to enterprise standards. \* Evaluate and recommend … emerging technologies. \* Develop reference architectures, solution blueprints, and best-practice playbooks for Databricks implementations. \* Architect scalable ETL/ELT pipelines using PySpark, Spark SQL, Delta Live Tables, and Databricks Workflows. \* Implement performance optimization strategies for notebooks, clusters, job orchestration, and data storage. \* Guide teams in building curated, governed data ...

deliver actionable, enterprise-grade business insights.Key ResponsibilitiesData Architecture & CDL Governance: Design, implement, and govern the enterprise Common Data Layer (CDL) on Azure Databricks using Delta Lake and L0–L1–L2 architectures. Establish data quality frameworks, schema evolution standards, and deduplication logic.Pipeline & Integration Engineering: Oversee the creation of scalable … connection of Azure Databricks data assets into robust, secure, and intuitive Power BI dashboards.Required Experience & SkillsTechnical ExpertiseCloud Data Platforms: Advanced knowledge of Azure Databricks, Delta Lake, Azure Data Factory (ADF), and SQL querying.SAP Ecosystem: Deep data discovery skills within SAP S/4HANA, SAP BW/HANA ...

will be a senior member of the engineering team, responsible for building large-scale Azure Data platforms, data models and working on the Delta Lake that underpins their Databricks AI platforms. This is a fully remote role, but you also have the option to work from multiple … Need: Strong experience as a Senior Azure Data Engineer An expert in Azure Databricks and Medallion Architecture Strong skills in PySpark, Python, Delta Lake and Unity Catalog Knowledge of Kimball data modelling, such as star schema, snowflake, etc. Strong SQL skills A flexible approach to getting things done ...

/ELT solutions and enterprise data pipelines. Experience with event-driven and messaging architectures. Data Platforms & Cloud Technologies Hands-on experience with: Azure Data Lake Databricks Snowflake Delta Lake Azure Synapse Analytics/Microsoft Fabric Spark/PySpark Strong understanding of: Data Lake and Lakehouse architecture ...

Medallion Architecture, and ensuring our Databricks platform is secure, governed, and production ready.Key Responsibilities: Design, build, and optimise scalable data pipelines using Databricks, Delta Live Tables, Synapse, and Delta Lake.Develop ingestion, transformation, and orchestration workflows across diverse sources (APIs, SQL, SFTP, SaaS platforms).Implement strong data quality, lineage … across pipelines and clusters.Contribute to standards for versioning, DevOps, CI/CD, and testing for Databricks notebooks, workflows, and SQL assets.Support the rollout of Delta Sharing, Unity Catalog governance, and Iceberg/Delta interoperability where required.Partner with analysts and data scientists to ensure that high-quality, well-modelled ...

Bash for engaging credibly with engineering teams on data product implementation Familiarity with cloud data architectures across Azure and AWS, including Azure Data Lake and Delta Lake Understanding of the broader data/infra toolchain: Terraform, GitHub Actions, Azure DevOps, and Azure Monitor Knowledge of AI/ ...

including extensive experience with workflow orchestration tools (e.g., Apache Airflow, Dagster).Hands-on expertise with cloud data warehouses (e.g., Snowflake, BigQuery, Redshift) and data lake architectures (e.g., Databricks, Delta Lake, Apache Iceberg).Advanced SQL skills and proficiency in Python. Strong background in modern software development practices (testing ...

sound judgment to tradeoffs around performance, cost, reliability, and maintainability. What you'll do Design, build, and operate production ETL pipelines on Databricks using Delta Live Tables and Spark, applying medallion architecture patterns across Finance, Payments, Product, and GTM domains. Own data modeling in different layers — defining governed, reusable … years of hands‐on data engineering experience building and maintaining production pipelines and data models. Deep proficiency with Databricks or similar platforms — Delta Live Tables, Unity Catalog, Delta Lake, and PySpark or Spark SQL as primary tools. Strong SQL skills: complex transformations, incremental materialization, schema evolution ...

workflows Excellent communication and collaboration skills in agile, cross-functional teams Ability to mentor junior engineers Nice to have: Experience with Databricks, PySpark, and Delta Lake Условия Competitive salary based on location, experience, and skills Comprehensive benefits package for employees and their families Continuous learning, structured development programs ...

robust ETL/ELT solutions using Azure Data Factory and Microsoft Fabric. Develop high-performance data processing solutions using Python, PySpark, and SQL. Implement Delta Lake and Lakehouse architectures following industry best practices. Work with Azure Data Lake Storage (ADLS Gen2) to manage large-scale datasets. Optimise ...

communicate complex concepts in a clear, concise manner• Languages: Terraform, Python, SQL, PowerShell.• CI/CD: Azure DevOps pipelines for data platform deployments.• Databricks: Delta Lake, Unity Catalog, Workflows, Notebooks, Clusters, MLflow.• Azure: ADLS Gen2, Azure Data Factory, Event Hubs/Kafka, Key Vault, Functions, Synapse. AI Foundry … Service Plan Deployment and Configuration, Integration with Azure bot Services• MS Fabric: Tenant Management, Workspace Private Links, Deployment, Capacity Management, Workspace Management, One Lake, Data Mirroring, Power BI• AI Foundry : Model management, Hosted Agents Deployment and Management , Project Configuration (Connections and Governance ) and lifecycle management.#LI-PM1Inclusion & DiversityWe value diversity ...

communicate complex concepts in a clear, concise manner Languages: Terraform, Python, SQL, PowerShell. CI/CD: Azure DevOps pipelines for data platform deployments. Databricks: Delta Lake, Unity Catalog, Workflows, Notebooks, Clusters, MLflow. Azure: ADLS Gen2, Azure Data Factory, Event Hubs/Kafka, Key Vault, Functions, Synapse. AI Foundry … Service Plan Deployment and Configuration, Integration with Azure bot Services MS Fabric: Tenant Management, Workspace Private Links, Deployment, Capacity Management, Workspace Management, One Lake, Data Mirroring, Power BI AI Foundry : Model management, Hosted Agents Deployment and Management , Project Configuration (Connections and Governance ) and lifecycle management. Inclusion & Diversity We value ...

term platform innovation Required Experience Strong experience designing enterprise data platforms within Microsoft Azure environments Deep expertise with Azure Databricks, Spark (PySpark/Scala), Delta Lake, and Lakehouse architecture Strong hands-on experience with Snowflake architecture and optimisation Experience with Azure Data Factory, ADLS Gen2, Synapse Analytics ...

such as Snowflake's Snowpark, Dynamic Tables, Streams & Tasks, and Snowflake Cortex, as well as Databricks components such as Lakeflow (Connect, Pipelines, and Jobs), Delta Lake, and Unity Catalog. Strong data engineering fundamentals: ETL/ELT pipeline design and implementation, data orchestration and workflow automation, batch and streaming ...

systematic approach to diagnosing and resolving issuesHighly desirable skillsExperience with streaming frameworks such as Flink, Kafka Streams and Kafka ConnectKnowledge of modern data lake technologies including Delta Lake, Iceberg and Glue Data CatalogExposure to DataOps practices and collaboration with Data Engineering teamsFamiliarity with GPU-accelerated analytics using ...

environment strategy. Design and govern data integration patterns for sharing data with external clients and third parties, including secure, scalable approaches using Databricks Delta Sharing, APIs, and file-based exchange where appropriate. Lead the design and adoption of Master Data Management (MDM), including entity definitions, golden-record strategy, match … merge rules, and stewardship processes. Establish standards and practices for optimizing Databricks performance and cost — cluster and compute strategy, Delta table management, caching, and workload right-sizing. Define data modelling standards (conceptual, logical, physical) across analytical and operational domains. Collaborate with data governance to embed data quality, lineage, cataloguing ...