16 of 16 Delta Lake Jobs in the UK

Data Governance, Data Quality and Analytics colleagues. The role will help translate platform requirements into practical modelling patterns across Azure Data Factory, Azure Databricks, Delta Lake, Unity Catalog, Microsoft Purview and Power BI, including metadata, lineage, data quality and certification requirements. These activities call for individuals … bronze, silver and gold data layers within a medallion lakehouse architecture. Working with Data Engineers to translate logical and dimensional models into implementable Delta Lake structures, including keys, relationships, history, conformed dimensions and data product boundaries. Supporting the definition of metadata, lineage, classification, glossary and certification requirements across ...

team builds to. Responsibilities Design and own the end-to-end Databricks architecture, from ingestion through to consumption Define batch and streaming patterns using Delta Lake, Delta Live Tables and Workflows Lead data modelling and lakehouse design Implement governance through Unity Catalog, including access control, lineage … engineering designs and mentor the team Translate business and regulatory requirements into technical solutions Essential Proven Data Architect experience designing production Databricks platforms Strong Delta Lake, Spark (PySpark/SQL) and Unity Catalog skills Financial services background, ideally with regulatory data (BCBS 239, GDPR, FCA/PRA) Data ...

scalable data pipelines using Dataflow Gen2 and Fabric Notebooks • Develop and maintain Lakehouse solutions using Medallion architecture • Work with SQL, Python, Spark SQL and Delta Lake • Integrate data from APIs, databases, CRM, ERP and third party platforms • Support both batch and real time data processing • Develop semantic models … Notebooks and Lakehouse architecture • Strong SQL and Python skills • Experience with Spark and Spark SQL • Proven experience designing scalable ETL and ELT pipelines • Data Lake and Delta Lake experience • Experience integrating data from multiple structured and unstructured sources • Good understanding of CI/CD, DevOps and modern ...

enterprise environments 4+ years of hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform Strong expertise in Apache Spark, SQL, Python, Delta Lake, and modern data engineering practices Experience building and optimizing Extract, Transform, Load and Extract, Load, Transform data pipelines for scalable analytics … Platform. Design and implement scalable data and analytics solutions leveraging Databricks technologies Build, maintain, and optimize data pipelines using Apache Spark, SQL, Python, and Delta Lake Develop governed datasets and data products that support reporting, analytics, and artificial intelligence initiatives Partner with business stakeholders and technical teams ...

build and enhance a secure, scalable and reliable Databricks platform Develop and maintain batch and real-time data pipelines using Databricks, Apache Spark and Delta Lake Contribute to platform architecture, engineering standards and best practices across the data estate Work with Product, Analytics and Data Science teams … across the engineering team Job Requirements: Significant experience building and supporting scalable, production-grade data platforms Strong hands-on expertise with Databricks, Apache Spark, Delta Lake, Python and SQL Good understanding of cloud-based data engineering, ideally within Microsoft Azure environments Experience with Infrastructure as Code tools such ...

ongoing enhancement and modernisation of enterprise data platforms Databricks & Data Platform Engineering Develop and support Databricks-based data solutions and Lakehouse architectures Utilise Delta Lake and Spark technologies to deliver scalable data processing capabilities Optimise distributed data processing workloads for performance and reliability Implement data quality controls ...

level discussion and technical detail Hands on implementation experience, ideally from a data engineering or developer background Databricks at architecture level on Azure, including Delta Lake, Unity Catalog and Spark optimisation Lakehouse and medallion patterns, batch and streaming ingestion, event driven design Conceptual, logical and physical data modelling. … Dimensional and data vault both useful Advanced SQL, Python or PySpark Declarative pipelines (Delta Live Tables or equivalent), Databricks Workflows, Auto Loader and Structured Streaming Microsoft Purview for data governance, cataloguing and lineage Azure Data Factory, Azure SQL, Data Lake Storage Gen2 Azure DevOps or GitHub Actions, Databricks ...

extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise data architecture, balancing … growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions. ...

data governance frameworks, metadata management, and regulatory complianc It would be a bonus if you have: Familiarity with emerging technologies such as Apache Iceberg, Delta Lake, real-time/streaming architectures, or data observability tooling Experience leading architecture workshops with both technical and non-technical stakeholders A background ...

data fidelity. Collaborate with cross-functional teams to align data architecture with financial regulations (e.g., GDPR, Basel III, MiFID II). Drive adoption of Delta Lake , Apache Spark , and MLflow for advanced analytics and machine learning use cases. Provide technical leadership and mentoring to data engineers and analysts ...

Databricks and Fabric architecture, including Lakehouse patterns Ability to oversee ingestion, transformation and reporting layers across enterprise data flows Working knowledge of Databricks (Delta Lake, pipelines, orchestration) and Fabric (Power BI, semantic models) Experience in operating appropriate governance over data quality, lineage, access and cross-border data controls ...

+ performance troubleshooting. Supporting + maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis + permanent remediation. Experience of Spark SQL, Delta Lake, Parquet + Hive metastore tables, including schemas, partitions and the relationship between table metadata and underlying files. Hands-on AWS experience, particularly ...

+ performance troubleshooting. Supporting + maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis + permanent remediation. Experience of Spark SQL, Delta Lake, Parquet + Hive metastore tables, schemas, partitions and the relationship between table metadata and underlying files. Hands-on AWS experience, particularly ...

design and pushing for better practice What youll need Proven data modelling experience (dimensional, 3NF, data vault or a mix) Hands-on Databricks and Delta Lake Solid understanding of medallion architecture, built in the real world Strong SQL Financial services or capital markets background is a big plus ...

pushing for better practice What you'll need Proven data modelling experience (dimensional, 3NF, data vault or a mix) Hands-on Databricks and Delta Lake Solid understanding of medallion architecture, built in the real world Strong SQL Financial services or capital markets background is a big plus Nice ...

pushing for better practice What you'll need Proven data modelling experience (dimensional, 3NF, data vault xehkeey or a mix) Hands-on Databricks and Delta Lake Solid understanding of medallion architecture, built in the real world Strong SQL Financial services or capital markets background is a big plus ...