5 of 5 Contract Delta Lake Jobs in the UK

lead engineer experience on enterprise scale Databricks platforms on Azure Strong hands on data engineering background with Python or PySpark and advanced SQL Delta Lake, Unity Catalog, medallion architecture, Databricks Workflows and Auto Loader Declarative pipelines (Delta Live Tables or Lakeflow) and Structured Streaming CI/… Databricks using Databricks Asset Bundles, Azure DevOps or GitHub Actions, and Terraform or Bicep Azure Data Factory, Azure SQL, Data Lake Storage Gen2 Entra ID, Key Vault, private endpoints, RBAC Experience of data governance and lineage, ideally with Microsoft Purview UK GDPR and DPA 2018 in practice, including public ...

ongoing enhancement and modernisation of enterprise data platforms Databricks & Data Platform Engineering Develop and support Databricks-based data solutions and Lakehouse architectures Utilise Delta Lake and Spark technologies to deliver scalable data processing capabilities Optimise distributed data processing workloads for performance and reliability Implement data quality controls ...

level discussion and technical detail Hands on implementation experience, ideally from a data engineering or developer background Databricks at architecture level on Azure, including Delta Lake, Unity Catalog and Spark optimisation Lakehouse and medallion patterns, batch and streaming ingestion, event driven design Conceptual, logical and physical data modelling. … Dimensional and data vault both useful Advanced SQL, Python or PySpark Declarative pipelines (Delta Live Tables or equivalent), Databricks Workflows, Auto Loader and Structured Streaming Microsoft Purview for data governance, cataloguing and lineage Azure Data Factory, Azure SQL, Data Lake Storage Gen2 Azure DevOps or GitHub Actions, Databricks ...

teams, Power BI datasets or reporting solutions. A collaborative approach, good problem-solving skills and the ability to communicate technical concepts clearly. Experience with Delta Lake, Azure DevOps, CI/CD or ITIL-aligned environments would be beneficial. Exposure to finance, procurement, hospitality or retail data would also ...

+ performance troubleshooting. Supporting + maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis + permanent remediation. Experience of Spark SQL, Delta Lake, Parquet + Hive metastore tables, including schemas, partitions and the relationship between table metadata and underlying files. Hands-on AWS experience, particularly ...