26 to 50 of 141 Delta Lake Jobs in the UK

GitHub Actions, GIT, Artifactory, SonarPreferred Qualifications:Bachelor’s degree in Computer Science, Engineering, Mathematics or related disciplineExtensive knowledge of data architecture principles (e.g., Data Lake, Databricks Delta Lake, Data Warehousing, etc.)Extensive knowledge of data modelling techniques including slowly changing dimensions, aggregation, partitioning and indexing strategiesExperience working ...

scope to drive data democratisation, governance maturity and AI-readiness across the firm.## **What you will be doing** * Design and maintain scalable, secure data lake and lakehouse solutions using Azure Data Lake, Databricks and Delta Lake — built to support real-time insight, advanced analytics … validation, logging and monitoring to catch issues early and keep the platform reliable. **Key Requirements** * Experience in data engineering, including designing and delivering data lake or lakehouse architectures in an enterprise cloud environment, ideally Microsoft Azure.* Hands-on expertise with Databricks, Spark, PySpark, SQL and core Azure Data Services ...

writing complex high-performance queries, understanding how queries execute on different compute technologies Azure Data Factory or other cloud orchestration tools Azure Data Lake storage and Delta Lake Unity Catalog, Purview or other Data Governance tools IAC tool such as Terraform, Azure DevOps Other technology we like ...

support API, file-based and SFTP integration patterns Technical design preparation and structured testing support Preferred Qualifications Exposure to reporting or Power BI datasets Delta Lake or Delta MERGE knowledge Databricks Asset Bundles experience Azure DevOps experience MLOps or model deployment experience Knowledge of SFMC or Comarch ...

deployment Worked across monitoring, logging and operational alerting Completed data-quality and reconciliation controls API/file/SFTP integration patterns Nice to have: Delta Lake/Delta MERGE Databricks Asset Bundles Azure DevOps MLOps/model deployment work history SFMC or Comarch integration Customer decisioning/ ...

Must Have: At least 7 years as an Engineer in a Data Field Strong Azure data skills, including: Azure Data Factory V2 Azure Data Lake Storage V2 Azure Databricks Azure Function Apps& Logic Apps Azure Stream Analytics Azure Resource Manager skills (Terraform, Azure Portal, Az CLI and Az PowerShell … Strong PySpark, Delta Lake, Unity Catalog and Python skills. Includes ability to write unit and integration tests in Python with unittest, pytest, etc. Strong understanding of software development practices, such as SOLID principals, structuring code, testing, IOC, etc. Strong repositories, CI/CD skills. Strong knowledge of Kimball ...

extensive experience with workflow orchestration tools (e.g., Apache Airflow, Dagster). Hands-on expertise with cloud data warehouses (e.g., Snowflake, BigQuery, Redshift) and data lake architectures (e.g., Databricks, Delta Lake, Apache Iceberg). Advanced SQL skills and proficiency in Python. Strong background in modern software development practices ...

other teams can pick up and run, rather than as documents. Azure-First Data Platforms: Deep architecture and build experience on Microsoft Azure (Data Lake/Lakehouse patterns, ADLS Gen2, Synapse Analytics, Azure Databricks, Delta Lake), with enough hands-on AWS or Google Cloud Platform (GCP) experience … Lead Azure-First Platform Architecture: Architect and implement Azure data platforms for clients, including landing zones, networking, identity, and data platform services (Azure Data Lake Storage, Synapse, Databricks), extending patterns to AWS or GCP where a client estate requires it. Design & Build Data Pipelines: Design, Implement key data ingress ...

scalable, maintainable, and aligned with organisational standards and governance requirements. Design, build, and optimise data pipelines on Databricks using PySpark and Spark SQL, applying Delta Lake and a medallion/Lakehouse architecture. Implement data validation and reconciliation checks, and automated testing across pipelines to ensure trusted, accurate outputs. ...

Unifeye. What You'll Do Design and build scalable batch and streaming data pipelines using Spark and Databricks. Develop modern Lakehouse architectures using Delta Lake and Medallion (Bronze, Silver and Gold) design principles. Build robust ingestion frameworks from APIs, files and streaming data sources. Ensure data quality, testing … data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good understanding of Lakehouse architecture and Delta Lake. Experience working with AWS, Azure or GCP. Experience implementing CI/CD, version control and deployment automation. Strong communication skills and experience working ...

Unifeye. What You'll Do Design and build scalable batch and streaming data pipelines using Spark and Databricks. Develop modern Lakehouse architectures using Delta Lake and Medallion (Bronze, Silver and Gold) design principles. Build robust ingestion frameworks from APIs, files and streaming data sources. Ensure data quality, testing … data platforms using Databricks or an equivalent Spark-based platform. Strong SQL skills and experience with data modelling. Good understanding of Lakehouse architecture and Delta Lake. Experience working with AWS, Azure or GCP. Experience implementing CI/CD, version control and deployment automation. Strong communication skills and experience working ...

cloud platforms (AWS, Azure, or GCP) including Snowflake deployment, networking (Private Link), and cloud storage integration (S3, ADLS, GCS); familiarity with lakehouse stacks (Databricks, Delta Lake, Apache Iceberg) is a plus.Mastery of Snowflake security and governance: RBAC, tag-based governance, dynamic data masking, row access policies, column-level ...

such as Kafka or Kinesis* (Desirable) Experience with Airflow or other workflow orchestration tools* (Desirable) Experience with Docker and Kubernetes* (Desirable) Exposure to Data Lake and Lakehouse architecture* (Desirable) Experience in Agile/Scrum development environments* (Desirable) Experience with Apache Kafka, Apache Airflow, Terraform, Databricks, Delta Lake ...

Snowflake and cloud-native data services Data architecture:Lakehouse, medallion, datameshand domain-oriented patterns Data processing:Apache Spark,SQLand distributed processing technologies Data storage:Delta Lake, ApacheIcebergand open table formats Data pipelines:Enterprise-scale ETL/ELT,batchand streaming patterns Transformation and quality:dbt, Great Expectations and equivalent ...

architecture, and environment strategy.* Design and govern integration patterns for sharing data with external clients and third parties, including secure, scalable approaches using Databricks Delta Sharing, APIs, and file-based exchange where appropriate.* Lead the design and adoption of Master Data Management (MDM), including entity definitions, golden-record strategy … match-and-merge rules, and stewardship processes.* Establish standards and practices for optimising Databricks performance and cost — cluster and compute strategy, Delta table, caching, and workload right-sizing.* Define data modelling standards (conceptual, logical, physical) across analytical and operational domains.* Work with data governance colleagues to embed data quality ...

lead engineer experience on enterprise scale Databricks platforms on Azure Strong hands on data engineering background with Python or PySpark and advanced SQL Delta Lake, Unity Catalog, medallion architecture, Databricks Workflows and Auto Loader Declarative pipelines (Delta Live Tables or Lakeflow) and Structured Streaming CI/… Databricks using Databricks Asset Bundles, Azure DevOps or GitHub Actions, and Terraform or Bicep Azure Data Factory, Azure SQL, Data Lake Storage Gen2 Entra ID, Key Vault, private endpoints, RBAC Experience of data governance and lineage, ideally with Microsoft Purview UK GDPR and DPA 2018 in practice, including public ...

Experience migrating data pipelines or workloads from legacy ETL platforms to Databricks. Experience with Informatica or similar enterprise integration tooling. Experience with Azure Data Lake and related Azure data services. Practical experience of Delta Lake, medallion or Lakehouse architecture patterns. Experience with CI/CD, automated testing ...

Experience migrating data pipelines or workloads from legacy ETL platforms to Databricks.• Experience with Informatica or similar enterprise integration tooling.• Experience with Azure Data Lake and related Azure data services.• Practical experience of Delta Lake, medallion or Lakehouse architecture patterns.• Experience with CI/CD, automated testing ...

ongoing enhancement and modernisation of enterprise data platforms Databricks & Data Platform Engineering Develop and support Databricks-based data solutions and Lakehouse architectures Utilise Delta Lake and Spark technologies to deliver scalable data processing capabilities Optimise distributed data processing workloads for performance and reliability Implement data quality controls ...

diagnosing and resolving issues Highly Desirable Skills Experience with streaming frameworks such as Flink, Kafka Streams and Kafka Connect Knowledge of modern data lake technologies including Delta Lake, Iceberg and Glue Data Catalog Exposure to DataOps practices and collaboration with Data Engineering teams Familiarity with GPU‐accelerated ...

level discussion and technical detail Hands on implementation experience, ideally from a data engineering or developer background Databricks at architecture level on Azure, including Delta Lake, Unity Catalog and Spark optimisation Lakehouse and medallion patterns, batch and streaming ingestion, event driven design Conceptual, logical and physical data modelling. … Dimensional and data vault both useful Advanced SQL, Python or PySpark Declarative pipelines (Delta Live Tables or equivalent), Databricks Workflows, Auto Loader and Structured Streaming Microsoft Purview for data governance, cataloguing and lineage Azure Data Factory, Azure SQL, Data Lake Storage Gen2 Azure DevOps or GitHub Actions, Databricks ...

powered solutions and using tools like GitHub Copilot, Claude Code or Cursor day to day, with real examples Tech: Azure, ADF, Synapse, Databricks, Delta Lake, Spark, Terraform, Data Vault, LLM APIs, RAG, agentic frameworks If you're a UK-based data engineer with consultancy experience and a practical ...

Apache Kafka or Amazon MSK, including stream processing with Spark Structured Streaming. Hands‐on experience with open table formats, such as Apache Iceberg or Delta Lake, including partitioning, schema evolution and table maintenance at scale. Experience with pipeline orchestration and infrastructure‐as‐code using Terraform. Experience acting ...

batch, streaming, change data capture (CDC), ETL/ELT and orchestration. A good understanding of relational and non-relational databases, open table formats (e.g. Delta Lake, Apache Iceberg), and data storage and lifecycle design. Practical knowledge of data governance, including cataloguing, lineage, data quality, ownership, classification and access ...

least 3 years hands-on with Azure Databricks in production environments Proven experience designing and implementing medallion architecture (Bronze/Silver/Gold) using Delta Lake on Azure Databricks Strong hands-on experience with Unity Catalog — metastore setup, catalog and schema design, access controls, and governance policies Experience … configuring Azure Data Lake Storage Gen2 — containers, access control lists, lifecycle policies, and integration with Databricks Hands-on experience with Microsoft Fabric — specifically Mirrored Azure Databricks Unity Catalog and Power BI connectivity to Databricks SQL Warehouses Experience with enterprise data source integration — MS SQL Server (on-premises and Azure ...