1 to 25 of 36 Delta Lake Jobs in the UK excluding London

business. As part of this journey, we're consolidating our data capabilities onto a single Azure Databricks platform, leveraging technologies including Azure Data Factory, Delta Lake, Unity Catalog and Power BI to unlock new insights and drive innovation. Joining our Enterprise Data Team , you'll play … driven initiatives across the organisation. You'll be responsible for building and maintaining data pipelines using Azure Databricks, Azure Data Factory and Delta Lake, supporting the modernisation of legacy platforms while ensuring data quality, reliability and performance. The role will also involve implementing data governance and security controls ...

optimize enterprise-scale data platforms using the Databricks Lakehouse Platform . The ideal candidate will have deep expertise in Databricks architecture, Apache Spark, Delta Lake, data engineering, cloud platforms, and enterprise data solutions. Key Responsibilities Design and architect Databricks Lakehouse solutions for large-scaleenterprise data platforms. Lead … deployment, and optimization. Define Databricks workspace architecture, clusterstrategies, security models, and operational standards. Design and developscalable data pipelinesusing Databricks, Apache Spark, PySpark, SQL, andDelta Lake . Implement data ingestionframeworks supporting batchand real-timedata processing. Design Delta Lake architectures including: Delta tables Medallion architecture (Bronze/Silver ...

team builds to. Responsibilities Design and own the end-to-end Databricks architecture, from ingestion through to consumption Define batch and streaming patterns using Delta Lake, Delta Live Tables and Workflows Lead data modelling and lakehouse design Implement governance through Unity Catalog, including access control, lineage … engineering designs and mentor the team Translate business and regulatory requirements into technical solutions Essential Proven Data Architect experience designing production Databricks platforms Strong Delta Lake, Spark (PySpark/SQL) and Unity Catalog skills Financial services background, ideally with regulatory data (BCBS 239, GDPR, FCA/PRA) Data ...

scalable data pipelines using Dataflow Gen2 and Fabric Notebooks • Develop and maintain Lakehouse solutions using Medallion architecture • Work with SQL, Python, Spark SQL and Delta Lake • Integrate data from APIs, databases, CRM, ERP and third party platforms • Support both batch and real time data processing • Develop semantic models … Notebooks and Lakehouse architecture • Strong SQL and Python skills • Experience with Spark and Spark SQL • Proven experience designing scalable ETL and ELT pipelines • Data Lake and Delta Lake experience • Experience integrating data from multiple structured and unstructured sources • Good understanding of CI/CD, DevOps and modern ...

data from internal systems and third-party platforms including Salesforce, Sopra Banking Services, CACI and Experian. Engineer Modern Data Architectures - Design Lakehouse, Warehouse and Delta Lake solutions using medallion architectures to support enterprise analytics and reporting. Deliver Analytics-Ready Data Assets - Build trusted, well-modelled datasets and semantic … successfully built, enhanced or operated cloud‐based data platforms, ideally within Azure environments. Data Architecture Knowledge - You understand data modelling, ELT/ETL design, Delta Lake principles, data quality frameworks and modern engineering best practices. Cloud & DevOps Capability - You have experience with Azure services, automation, CI/ ...

thrive. Your day at NTT DATA Primary Responsibilities Client Engagement & Delivery Data Pipeline Development (Batch and Streaming) Databricks & Lakehouse Architectures Data Modelling & Optimisation (Delta Lake, Medallion architecture) Collaboration & Best Practices Quality, Governance & Security Business Relationships Solution Architects Data Engineers, Developers, ML Engineers, and Analysts Client stakeholders … mentoring teams of engineers to deliver high-quality scalable data solutions. Technical Expertise Deep expertise with the Databricks platform (Spark/PySpark/Scala, Delta Lake, Unity Catalog, MLflow). Proficiency in ETL/ELT tools such as DBT, Matillion, Talend, or equivalent. Strong SQL and Python ...

build and enhance a secure, scalable and reliable Databricks platform Develop and maintain batch and real-time data pipelines using Databricks, Apache Spark and Delta Lake Contribute to platform architecture, engineering standards and best practices across the data estate Work with Product, Analytics and Data Science teams … across the engineering team Job Requirements: Significant experience building and supporting scalable, production-grade data platforms Strong hands-on expertise with Databricks, Apache Spark, Delta Lake, Python and SQL Good understanding of cloud-based data engineering, ideally within Microsoft Azure environments Experience with Infrastructure as Code tools such ...

managing and reporting control issues with transparency. Skills & Experience Essential Proven in-depth commercial experience in the following key areas: Databricks – Lakehouse architecture, Spark, Delta Lake, workflows and performance tuning. Apache Spark & Scala – High-performance distributed data processing and optimisation. Cloud Data Engineering (AWS) – Cloud-native data platforms … core business functions Demonstrated leadership, project management, and development skills Relationship and consensus building skills Preferred Kafka/Event-Driven Architecture Apache Iceberg/Delta Lake Python AI/ML & LLM integration Financial Services/Capital Markets Education Bachelor’s degree/University degree or equivalent experience Master ...

build and optimise data pipelines across Pricing, Underwriting and Claims. You’ll work on a modern Azure-based data platform, consolidating capabilities on Databricks, Delta Lake and Unity Catalog to support analytics and AI initiatives. You’ll collaborate with Data Architects and Analysts, delivering scalable, secure data solutions ...

scalable, maintainable, and aligned with organisational standards and governance requirements. Design, build, and optimise data pipelines on Databricks using PySpark and Spark SQL, applying Delta Lake and a medallion/Lakehouse architecture. Implement data validation and reconciliation checks, and automated testing across pipelines to ensure trusted, accurate outputs. ...

cloud platforms (AWS, Azure, or GCP) including Snowflake deployment, networking (Private Link), and cloud storage integration (S3, ADLS, GCS); familiarity with lakehouse stacks (Databricks, Delta Lake, Apache Iceberg) is a plus.Mastery of Snowflake security and governance: RBAC, tag-based governance, dynamic data masking, row access policies, column-level ...

data platforms. Expertise with one or more of the following technologies: AWS (Redshift, Glue, S3, Bedrock) Microsoft Azure (Synapse, Data Factory, Azure SQL) Databricks (Delta Lake, ETL pipelines, ML workloads) Snowflake Strong data modelling experience across enterprise-scale environments. Experience building modern AI-enabled data architectures. Knowledge ...

data engineering, data platform delivery, software engineering, or related enterprise technology roles. Strong hands‐on experience with Databricks, Spark/PySpark, Python, SQL, Delta Lake, Azure, ADLS, and cloud‐based data engineering patterns. Experience designing scalable data pipelines, data products, and lakehouse solutions using modern cloud data platform ...

batch, streaming, change data capture (CDC), ETL/ELT and orchestration. A good understanding of relational and non‐relational databases, open table formats (e.g. Delta Lake, Apache Iceberg), and data storage and lifecycle design. Practical knowledge of data governance, including cataloguing, lineage, data quality, ownership, classification and access ...

least 3 years hands-on with Azure Databricks in production environments Proven experience designing and implementing medallion architecture (Bronze/Silver/Gold) using Delta Lake on Azure Databricks Strong hands-on experience with Unity Catalog — metastore setup, catalog and schema design, access controls, and governance policies Experience … configuring Azure Data Lake Storage Gen2 — containers, access control lists, lifecycle policies, and integration with Databricks Hands-on experience with Microsoft Fabric — specifically Mirrored Azure Databricks Unity Catalog and Power BI connectivity to Databricks SQL Warehouses Experience with enterprise data source integration — MS SQL Server (on-premises and Azure ...

extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise data architecture, balancing … growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions. ...

divergent source systems into canonical models across multiple products. Strong grounding in Lakehouse architecture trade-offs: table formats, query engines, ingestion patterns (Iceberg vs Delta; Dremio vs Trino vs Snowflake; streaming vs batch). Hands-on experience with: Dremio, Snowflake, or Databricks AWS data services (S3, Glue, Athena, Lake … Stream processing (Apache Flink, Spark Structured Streaming, or equivalent) SQL and query optimisation at scale Data and table formats (Parquet, Avro, Apache Iceberg or Delta Lake) Experience designing SaaS data models, tenant isolation, shared schemas, access patterns, at scale. Ability to communicate architecture decisions to engineers, product leadership ...

time streaming, and CI/CD pipelines (Git, Terraform, Databricks Asset Bundles). Deep knowledge of Databricks/Snowflake environments, including Unity Catalog, workspaces, Delta Lake, and networking/security (VPNs, Private Links, Firewalls) and/or Snowflake Platform including Cortex, Horizon and Snowpark. Advanced proficiency in Azure ...

time streaming, and CI/CD pipelines (Git, Terraform, Databricks Asset Bundles). Deep knowledge of Databricks/Snowflake environments, including Unity Catalog, workspaces, Delta Lake, and networking/security (VPNs, Private Links, Firewalls) and/or Snowflake Platform including Cortex, Horizon and Snowpark. Advanced proficiency in Azure ...

Databricks and Fabric architecture, including Lakehouse patterns Ability to oversee ingestion, transformation and reporting layers across enterprise data flows Working knowledge of Databricks (Delta Lake, pipelines, orchestration) and Fabric (Power BI, semantic models) Experience in operating appropriate governance over data quality, lineage, access and cross-border data controls ...

cloud databases Proven track record with complex data migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing Knowledge ...

cloud databases Proven track record with complex data migration projects (terabyte+ datasets, multiple legacy source systems, structures and unstructured data) Proficiency with Parquet/Delta Lake or other modern data storage formats Experience with streaming architectures using Kafka, Event Hubs, or Kinesis for real-time data processing Knowledge ...

+ performance troubleshooting. Supporting + maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis + permanent remediation. Experience of Spark SQL, Delta Lake, Parquet + Hive metastore tables, including schemas, partitions and the relationship between table metadata and underlying files. Hands-on AWS experience, particularly ...

+ performance troubleshooting. Supporting + maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis + permanent remediation. Experience of Spark SQL, Delta Lake, Parquet + Hive metastore tables, schemas, partitions and the relationship between table metadata and underlying files. Hands-on AWS experience, particularly ...

solid understanding of agile delivery practices including CI/CD, Application Resiliency, and Security. Preferred qualifications, capabilities, and skills Production experience with Databricks (Delta Lake, Unity Catalog, MLflow, Databricks Jobs) and workflow orchestration with Apache Airflow. Strong working knowledge of both supervised and unsupervised techniques, including tree-based ...