20 of 20 Delta Lake Jobs in the UK

scalable data pipelines using Dataflow Gen2 and Fabric Notebooks • Develop and maintain Lakehouse solutions using Medallion architecture • Work with SQL, Python, Spark SQL and Delta Lake • Integrate data from APIs, databases, CRM, ERP and third party platforms • Support both batch and real time data processing • Develop semantic models … Notebooks and Lakehouse architecture • Strong SQL and Python skills • Experience with Spark and Spark SQL • Proven experience designing scalable ETL and ELT pipelines • Data Lake and Delta Lake experience • Experience integrating data from multiple structured and unstructured sources • Good understanding of CI/CD, DevOps and modern ...

enterprise environments 4+ years of hands-on experience designing and implementing solutions on the Databricks Lakehouse Platform Strong expertise in Apache Spark, SQL, Python, Delta Lake, and modern data engineering practices Experience building and optimizing Extract, Transform, Load and Extract, Load, Transform data pipelines for scalable analytics … Platform. Design and implement scalable data and analytics solutions leveraging Databricks technologies Build, maintain, and optimize data pipelines using Apache Spark, SQL, Python, and Delta Lake Develop governed datasets and data products that support reporting, analytics, and artificial intelligence initiatives Partner with business stakeholders and technical teams ...

best practices across the enterprise data ecosystem. The role leads the design and delivery of scalable, secure, high-performing data products using Databricks, Delta Lake, Unity Catalog and Azure Data Services The role provides technical leadership, ensuring solutions are robust, cost-effective, highly available, and aligned to business … development. To ensure data dictionaries and models are robust and maintained on a regular basis. Technical Skill Expert-level Databricks experience Advanced PySpark development Delta Lake architecture and optimisation Databricks SQL Data warehouse Dimensional modelling Data Lake Databricks Genie Python development Key Skills, Experience and Knowledge Extensive ...

supporting colleagues. What you'll do ·Build and optimise pipelines using Azure Databricks, PySpark, SQL, ADF and ADLS Gen2. ·Develop governed lakehouse solutions using Delta Lake, medallion architecture and Unity Catalog. ·Own Power BI semantic models, DAX measures and row-level security, ensuring reporting is accurate, secure …/CD. ·Mentor colleagues and share good practice across the team. What you'll bring ·Strong production experience with Databricks, PySpark, Delta Lake and Unity Catalog. ·Hands-on Power BI expertise, including data modelling, DAX, RLS and performance optimisation. ·Strong SQL skills and experience with Azure data services. ...

lead engineer experience on enterprise scale Databricks platforms on Azure Strong hands on data engineering background with Python or PySpark and advanced SQL Delta Lake, Unity Catalog, medallion architecture, Databricks Workflows and Auto Loader Declarative pipelines (Delta Live Tables or Lakeflow) and Structured Streaming CI/… Databricks using Databricks Asset Bundles, Azure DevOps or GitHub Actions, and Terraform or Bicep Azure Data Factory, Azure SQL, Data Lake Storage Gen2 Entra ID, Key Vault, private endpoints, RBAC Experience of data governance and lineage, ideally with Microsoft Purview UK GDPR and DPA 2018 in practice, including public ...

ongoing enhancement and modernisation of enterprise data platforms Databricks & Data Platform Engineering Develop and support Databricks-based data solutions and Lakehouse architectures Utilise Delta Lake and Spark technologies to deliver scalable data processing capabilities Optimise distributed data processing workloads for performance and reliability Implement data quality controls ...

level discussion and technical detail Hands on implementation experience, ideally from a data engineering or developer background Databricks at architecture level on Azure, including Delta Lake, Unity Catalog and Spark optimisation Lakehouse and medallion patterns, batch and streaming ingestion, event driven design Conceptual, logical and physical data modelling. … Dimensional and data vault both useful Advanced SQL, Python or PySpark Declarative pipelines (Delta Live Tables or equivalent), Databricks Workflows, Auto Loader and Structured Streaming Microsoft Purview for data governance, cataloguing and lineage Azure Data Factory, Azure SQL, Data Lake Storage Gen2 Azure DevOps or GitHub Actions, Databricks ...

level discussion and technical detail Hands on implementation experience, ideally from a data engineering or developer background Databricks at architecture level on Azure, including Delta Lake, Unity Catalog and Spark optimisation Lakehouse and medallion patterns, batch and streaming ingestion, event driven design Conceptual, logical and physical data modelling. … Dimensional and data vault both useful Advanced SQL, Python or PySpark Declarative pipelines (Delta Live Tables or equivalent), Databricks Workflows, Auto Loader and Structured Streaming Microsoft Purview for data governance, cataloguing and lineage Azure Data Factory, Azure SQL, Data Lake Storage Gen2 Azure DevOps or GitHub Actions, Databricks ...

extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise data architecture, balancing … growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions. ...

extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise data architecture, balancing … growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions. ...

extensive benefits package. What you'll be doing Design, build and maintain scalable data pipelines and data products using Databricks , PySpark, Spark SQL and Delta Lake. Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems. Shape enterprise data architecture, balancing … growing data function. What you need to bring “Senior” level experience in data engineering, with strong hands-on Databricks skills (PySpark, Spark SQL, Delta Lake, etc) Advanced SQL and Python coding skills. A track record of building and optimising enterprise-scale data pipelines and ETL/ELT solutions. ...

teams, Power BI datasets or reporting solutions. A collaborative approach, good problem-solving skills and the ability to communicate technical concepts clearly. Experience with Delta Lake, Azure DevOps, CI/CD or ITIL-aligned environments would be beneficial. Exposure to finance, procurement, hospitality or retail data would also ...

secure handling of inputs/outputs, and adherence to resiliency and security expectations Preferred qualifications, capabilities, and skills Experience with data lakehouse architectures using Delta Lake, Apache Spark, or similar open-source frameworks Familiarity with AWS-native data services such as Glue, Lake Formation, Redshift, or Athena ...

data governance frameworks, metadata management, and regulatory complianc It would be a bonus if you have: Familiarity with emerging technologies such as Apache Iceberg, Delta Lake, real-time/streaming architectures, or data observability tooling Experience leading architecture workshops with both technical and non-technical stakeholders A background ...

Excellent stakeholder management and communication skills. Experience providing technical leadership or mentoring. Desirable: Experience with Unity Catalog, dbt or similar transformation frameworks. Knowledge of Delta Lake, data governance and metadata management. Experience with CI/CD and automated testing. ...

Advanced Python, PySpark and SQL Production experience with Kafka or Amazon MSK, including Spark Structured Streaming Hands-on experience with Apache Iceberg or Delta Lake Strong data modelling and lakehouse design (bronze, silver, gold) Terraform, CI/CD and pipeline orchestration A track record of setting standards ...

Advanced Python, PySpark and SQL Production experience with Kafka or Amazon MSK, including Spark Structured Streaming Hands-on experience with Apache Iceberg or Delta Lake Strong data modelling and lakehouse design (bronze, silver, gold) Terraform, CI/CD and pipeline orchestration A track record of setting standards ...

Databricks and Fabric architecture, including Lakehouse patterns Ability to oversee ingestion, transformation and reporting layers across enterprise data flows Working knowledge of Databricks (Delta Lake, pipelines, orchestration) and Fabric (Power BI, semantic models) Experience in operating appropriate governance over data quality, lineage, access and cross-border data controls ...

+ performance troubleshooting. Supporting + maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis + permanent remediation. Experience of Spark SQL, Delta Lake, Parquet + Hive metastore tables, including schemas, partitions and the relationship between table metadata and underlying files. Hands-on AWS experience, particularly ...

+ performance troubleshooting. Supporting + maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis + permanent remediation. Experience of Spark SQL, Delta Lake, Parquet + Hive metastore tables, schemas, partitions and the relationship between table metadata and underlying files. Hands-on AWS experience, particularly ...