26 to 50 of 179 Delta Lake Jobs in the UK

practices, and guiding engineering teams to deliver scalable, secure, and cost optimized data platforms. \* Design end to end data platforms leveraging Databricks Lakehouse architecture, Delta Lake, Unity Catalog, and ML/AI workloads. \* Define data ingestion, transformation, governance, and consumption frameworks aligned to enterprise standards. \* Evaluate and recommend … emerging technologies. \* Develop reference architectures, solution blueprints, and best-practice playbooks for Databricks implementations. \* Architect scalable ETL/ELT pipelines using PySpark, Spark SQL, Delta Live Tables, and Databricks Workflows. \* Implement performance optimization strategies for notebooks, clusters, job orchestration, and data storage. \* Guide teams in building curated, governed data ...

Senior Data Engineer (Databricks) - London (Hybrid) - Up to £85,000 Databricks - Apache Spark - Delta Lake - Python - SQL - Azure We're partnering with a fast-growing technology business that's investing heavily in its data platform and engineering capability. This is an opportunity to join a highly collaborative team … growing technology business. Key Experience Required Proven experience as a Data Engineer within production environments. Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python, and SQL. Experience building and supporting scalable ETL/ELT pipelines. Strong understanding of distributed data processing and data modelling. Cloud platform ...

Databricks Senior Data Engineer - London (Hybrid) - Up to £85,000 Databricks - Apache Spark - Delta Lake - Python - SQL - Azure We're partnering with a fast-growing technology business that's investing heavily in its data platform and engineering capability. This is an opportunity to join a highly collaborative team … growing technology business. Key Experience Required Proven experience as a Data Engineer within production environments. Strong hands-on experience with Databricks, Apache Spark, Delta Lake, Python, and SQL. Experience building and supporting scalable ETL/ELT pipelines. Strong understanding of distributed data processing and data modelling. Cloud platform ...

shared data assets Technical Leadership & Standards Serve as a senior technical authority for data product architecture across the program Define architectural best practices for Delta Lake design, metadata management, product lifecycle governance, and data quality controls Provide architecture assurance and design reviews for data products developed by internal … data contracts, service models, and consumer engagement frameworks Excellent stakeholder management, communication, and consulting skills Technical Skills Deep expertise in Databricks, including Lakehouse Architecture, Delta Lake, Unity Catalog, and Databricks SQL Strong understanding of data modelling, schema design, metadata management, and lineage tracking Experience implementing self-service data ...

will be a senior member of the engineering team, responsible for building large-scale Azure Data platforms, data models and working on the Delta Lake that underpins their Databricks AI platforms. This is a fully remote role, but you also have the option to work from multiple … What You'll Need: Strong experience as a Senior Azure Data EngineerAn expert in Azure Databricks and Medallion Architecture Strong skills in PySpark, Python, Delta Lake and Unity CatalogKnowledge of Kimball data modelling, such as star schema, snowflake, etc. Strong SQL skillsA flexible approach to getting things done ...

regularly observe cloud hardware, network, and operating system faults, and our software must gracefully shield our customers from any of the above. The Delta DML team owns the core write-path operations for Delta Lake, the open-source storage layer behind the Databricks Lakehouse. Our mission … flowing through our platform. We drive performance innovations like Low Shuffle Merge and Deletion Vectors and actively contribute to open source efforts to unify Delta and Iceberg formats. We are seeking a highly skilled and experienced Senior Staff Software Engineer to join our backend team. In this role ...

managing and reporting control issues with transparency. Skills & Experience Essential Proven in-depth commercial experience in the following key areas: Databricks – Lakehouse architecture, Spark, Delta Lake, workflows and performance tuning. Apache Spark & Scala – High-performance distributed data processing and optimisation. Cloud Data Engineering (AWS) – Cloud-native data platforms … core business functions Demonstrated leadership, project management, and development skills Relationship and consensus building skills Preferred Kafka/Event-Driven Architecture Apache Iceberg/Delta Lake Python AI/ML & LLM integration Financial Services/Capital Markets Education Bachelor’s degree/University degree or equivalent experience Master ...

reporting. You’ll work on: Building and orchestrating ingestion pipelines for complex financial data Designing scalable data models across relational, NoSQL, blob and Delta Lake architectures Developing production APIs and data services for the wider SaaS platform Transforming and reconciling inconsistent structured and unstructured data Improving data quality … Care about data quality, clean engineering and genuine ownership Have experience building SaaS products or other production platforms Experience with Azure, Node.js, data warehousing, Delta Lake or vector databases would be valuable. You don’t need previous private-markets experience. What matters is strong engineering judgement ...

Azure Data Engineer with strong experience of Azure Databricks to join our data team. You will help build large-scale Azure data platforms and Delta Lake foundations.This fully remote role offers optional office access and the chance to work closely with the Head of Data on Databricks ...

will be a senior member of the engineering team, responsible for building large-scale Azure Data platforms, data models and working on the Delta Lake that underpins their Databricks AI platforms. This is a fully remote role, but you also have the option to work from multiple … Need: Strong experience as a Senior Azure Data Engineer An expert in Azure Databricks and Medallion Architecture Strong skills in PySpark, Python, Delta Lake and Unity Catalog Knowledge of Kimball data modelling, such as star schema, snowflake, etc. A flexible approach to getting things done with a focus ...

build and optimise data pipelines across Pricing, Underwriting and Claims. You’ll work on a modern Azure-based data platform, consolidating capabilities on Databricks, Delta Lake and Unity Catalog to support analytics and AI initiatives. You’ll collaborate with Data Architects and Analysts, delivering scalable, secure data solutions ...

than operating purely at a strategic level. Ideally, your experience will include several of the following: Strong commercial experience with Databricks Unity Catalog, notebooks, Delta Lake or Delta Live Tables Data lakehouse and medallion architecture Advanced data modelling, including relational, dimensional, Data Vault or similar approaches Python ...

batch data ingestion solutions using Azure Event Hubs, Kafka, Databricks Structured Streaming and Azure Stream Analytics Developing and maintaining modern data platforms, including Delta Lake/Lakehouse architectures on ADLS Gen2, curated datasets, dimensional models and Azure Synapse Dedicated SQL Pools Analysing current business practices, processes and procedures … essential with a regulated environment is essential Significant experience with data solution design experience in an MI/BI/Analytics environment (Kimball, lake house, data lake) is essential Significant experience using: Azure Data Factory, Azure Synapse Analytics, Azure SQL, Azure Data Lake, Datra bricks, Purview ...

Medallion Architecture, and ensuring our Databricks platform is secure, governed, and production ready. Key Responsibilities Design, build, and optimise scalable data pipelines using Databricks, Delta Live Tables, Synapse, and Delta Lake. Develop ingestion, transformation, and orchestration workflows across diverse sources (APIs, SQL, SFTP, SaaS platforms). Implement strong … clusters. Contribute to standards for versioning, DevOps, CI/CD, and testing for Databricks notebooks, workflows, and SQL assets. Support the rollout of Delta Sharing, Unity Catalog governance, and Iceberg/Delta interoperability where required. Partner with analysts and data scientists to ensure that high-quality, well‐modelled ...

Medallion Architecture, and ensuring our Databricks platform is secure, governed, and production ready. Key Responsibilities: Design, build, and optimise scalable data pipelines using Databricks, Delta Live Tables, Synapse, and Delta Lake. Develop ingestion, transformation, and orchestration workflows across diverse sources (APIs, SQL, SFTP, SaaS platforms). Implement strong … clusters. Contribute to standards for versioning, DevOps, CI/CD, and testing for Databricks notebooks, workflows, and SQL assets. Support the rollout of Delta Sharing, Unity Catalog governance, and Iceberg/Delta interoperability where required. Partner with analysts and data scientists to ensure that high-quality, well-modelled ...

Must Have At least 7 years as an Engineer in a Data Field Strong Azure data skills, including: Azure Data Factory V2 Azure Data Lake Storage V2 Azure Databricks Azure Function Apps& Logic Apps Azure Stream Analytics Azure Resource Manager skills (Terraform, Azure Portal, Az CLI and Az PowerShell … Strong PySpark, Delta Lake, Unity Catalog and Python skills. Includes ability to write unit and integration tests in Python with unittest, pytest, etc. Strong understanding of software development practices, such as SOLID principals, structuring code, testing, IOC, etc. Strong repositories, CI/CD skills. Strong knowledge of Kimball ...

including extensive experience with workflow orchestration tools (e.g., Apache Airflow, Dagster).Hands-on expertise with cloud data warehouses (e.g., Snowflake, BigQuery, Redshift) and data lake architectures (e.g., Databricks, Delta Lake, Apache Iceberg).Advanced SQL skills and proficiency in Python. Strong background in modern software development practices (testing ...

Skills Strong experience designing and delivering modern data platforms using Microsoft Fabric, Azure Databricks, Azure Data Factory, Azure Synapse Analytics, Azure SQL and Data Lake technologies . Proven experience implementing Lakehouse, Data Warehouse and Medallion Architecture patterns, including data modelling, governance and performance optimization . Expertise in building scalable … Python, Spark and PySpark for data engineering, analytics and automation workloads. Experience with Azure Databricks and/or Microsoft Fabric engineering workloads, including notebooks, Delta Lake, data transformation, optimization and operationalization of data solutions. An understanding of data governance, security, lineage and data quality, using tools and frameworks ...

systems design within a modern cloud‐based environment Advanced experience working and understanding the tradeoffs of at least one of the following Data Lake table/file formats: Delta Lake, Parquet, Iceberg, Hudi Experience working with lakehouse/medallion architectures in Databricks Experience with experimentation support tooling ...

customer in Central London. Key Responsibilities Data Lakehouse Architecture: Design and optimize a high-performance lakehouse pattern utilizing Microsoft Fabric OneLake and Azure Data Lake Storage Gen2 (ADLS Gen2). Data Zone Management: Maintain architectural segregation between raw storage (immutability, landing) and curated data zones (cleansed, conformed business layers … OneLake, Data Factory, Fabric Lakehouse). Big Data Processing: Deep technical expertise with Apache Spark/Azure Databricks for distributed compute, memory optimization, and Delta Lake formats. Data Engineering Languages: Advanced proficiency in SQL (T-SQL, SparkSQL) and Python (PySpark). Security & Compliance: Direct experience implementing security controls ...

managing complex datasets within a recognised database environment (preferably PostgreSQL/PostGIS) and/or a cloud storage facility (such as a Lakehouse, Delta Lake or Data Lake solution) Excellent organisational skills and the ability to manage competing priorities Experience of using GIS systems, such as ArcGIS ...

large scaleStrong programming skills in Python, with applied experience in data engineering workflowsExpertise in PySpark for distributed data processingHands-on experience with Databricks, including Delta Lake and performance optimizationKnowledge of Azure Data Factory for orchestration of pipelines (ETL/ELT)Familiarity with AI/ML pipeline development, including ...

data platforms. Expertise with one or more of the following technologies: AWS (Redshift, Glue, S3, Bedrock) Microsoft Azure (Synapse, Data Factory, Azure SQL) Databricks (Delta Lake, ETL pipelines, ML workloads) Snowflake Strong data modelling experience across enterprise-scale environments. Experience building modern AI-enabled data architectures. Knowledge ...

native data services Data architecture: Lakehouse, medallion, data mesh and domain-oriented patterns Data processing: Apache Spark, SQL and distributed processing technologies Data storage: Delta Lake, Apache Iceberg and open table formats Data pipelines: Enterprise-scale ETL/ELT, batch and streaming patterns Transformation and quality: dbt, Great ...

native data services Data architecture: Lakehouse, medallion, data mesh and domain-oriented patterns Data processing: Apache Spark, SQL and distributed processing technologies Data storage: Delta Lake, Apache Iceberg and open table formats Data pipelines: Enterprise-scale ETL/ELT, batch and streaming patterns Transformation and quality: dbt, Great ...