18 of 18 Delta Lake Jobs in the UK excluding London

reliable delivery of new data pipelines alongside ongoing operational stability.This is a hands-on engineering role focused on implementation, optimisation and support of Delta Lake pipelines across Bronze, Silver, Staging and Gold layers, operating within an established architectural framework. Key Responsibilities Develop and maintain data pipelines in Azure … Databricks using PySpark and SQL Build incremental processing using Delta Lake and Change Data Feed Extend and maintain watermark-driven CDC frameworks Implement and maintain Slowly Changing Dimension (SCD Type 2) logic Engineer time-aware joins and dimensional transformations Support optimisation of Spark workloads and Delta tables ...

working on behalf of a major client in the energy sector. The position focuses on designing, developing, and maintaining Azure-based data platforms-particularly Delta Lake and modern analytics services-while providing architectural leadership and hands-on engineering expertise. The role operates on a hybrid model, requiring three … days per week onsite in Glasgow. Key Responsibilities Develop and maintain Azure data architecture , including Delta Lake, ensuring compliance with data governance and security standards. Analyse and resolve data integration and system interoperability challenges across complex environments. Design and implement Azure data solutions using services such as Databricks ...

decisions, and working with people who care about building things properly, you'll fit right in. You'll Work With Azure Data Factory, Data Lake, Key Vault, Azure Functions Databricks, Delta Lake, Unity Catalog, PySpark, Spark SQL Medallion architecture, Lakehouse patterns, distributed compute CI/CD pipelines … home You'll thrive in this role if you have: Strong hands‐on experience with Azure Data Services and Databricks Solid understanding of Delta Lake, Medallion architecture and Lakehouse patterns Proficiency in Python, PySpark and Spark SQL Experience building CI/CD pipelines using Azure DevOps Knowledge ...

data becomes a true competitive advantage. Why consider this role? Lead delivery of a strategic cloud data platform Own architecture across Azure, Databricks, Synapse & Delta Lake Influence enterprise-wide data governance and standards Hybrid working - 3 days per week in the office Competitive salary, bonus & strong benefits High … regulated finance environment What we're looking for Proven experience as a Lead/Principal Data Engineer Deep Azure data stack knowledge (Databricks, Synapse, Delta Lake, Unity Catalog) Strong SQL and data-modelling capability Experience with orchestration, monitoring and data quality tooling Background working in regulated financial services ...

forward-thinking, data-driven organisation. If you thrive bridging the gap between data engineering and analytics, and enjoy working with tools like Databricks, Delta Lake, Spark, and dbt, this is an exciting short-term opportunity. What You'll Do: Transform raw data into structured, reliable datasets … workflows, SQL queries, and Databricks pipelines for performance. What We're Looking For: Strong SQL and data modelling skills. Hands-on experience with Databricks, Delta Lake, Spark, and preferably dbt. Python for data manipulation and automation. Familiarity with BI tools (Power BI) and cloud platforms (Azure). Understanding ...

business. Contribute to architectural discussions and the ongoing data platform roadmap. Tech You'll Use Databricks: Lakeflow Declarative Pipelines, Lakeflow Workflows, Unity Catalog, Delta Lake Azure: ADLS Gen2, Data Factory, Event Hubs (optional), Key Vault, private endpoints Languages: PySpark, Spark SQL, Python, Git DevOps: Azure DevOps Repos & Pipelines … Azure + Databricks . Strong PySpark and Spark SQL skills within distributed compute environments. Experience working in a Lakehouse/Medallion architecture with Delta Lake. Understanding of dimensional modelling (Kimball), including SCD Type 1/2. Exposure to operational concepts such as monitoring, retries, idempotency and backfills. Mindset Good ...

line with the organisation’s norms and values Essential • Proven experience in managing data operations, integrations, and engineering teams. • Strong expertise in Databricks, including Delta Lake, Spark, and automated pipeline development. • Hands-on experience with ETL/ELT processes, data governance, and cloud-based data platforms. • Proficiency ...

pipelines using Azure Data Factory and Databricks. Develop scalable data models and transformations using SQL and Python . Work hands‐on with Databricks (Lakehouse, Delta tables, notebooks, workflows). Deliver high-quality dashboards and reporting solutions using Power BI . Implement best practices for data quality, governance, lineage … cost optimisation and reliability improvements across the data estate. Document pipelines, models and processes to ensure smooth knowledge transfer. Technical Skills Required Databricks – notebooks, Delta Lake, Spark (PySpark desirable) Power BI – data modelling, DAX, dashboard/report development SQL – advanced querying, performance optimisation, data modelling Python – scripting, transformation ...

cloud data strategy, define engineering best practice and guide delivery across cross-functional teams. Responsibilities Build and optimise Azure data platforms using ADF, Data Lake, Azure Functions and Databricks Deliver reliable, high-quality data pipelines Work with stakeholders to ensure solutions meet Lloyd's of London regulatory, audit … Data Services and Databricks in enterprise environments. Proven delivery experience within the Lloyd's of London or London Market insurance sector Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns and data modelling. Strong Python, PySpark and Spark SQL capability Solid understanding of data governance Proven track record ...

across the organisation. You'll design and develop scalable data pipelines within a modern Azure ecosystem, working with technologies such as Azure Synapse, Databricks, Delta Lake, Unity Catalog, and Databricks SQL. Responsibilities include overseeing pipeline orchestration, implementing data quality controls, monitoring and alerting, and managing data lineage, glossaries ...

wont just be delivering features; youll be setting direction. The tech youll work with Core:Azure, Databricks, Python, SQL, dbt, MQTT Storage & Serving:Delta Lake, Postgres, TimescaleDB Modelling & ML:MLflow Visualisation:Grafana What makes you a great fit Youre someone who can zoom out to architecture and zoom ...

streamlined data analysis, product builds and reduced time to market Competencies and Behaviours 1 -3 years analytics/data science experience Practical knowledge of; Delta Lake architecture and versioned datasets, Data pipelines, orchestration and scheduling concepts Proficiency in analytical programming language such as python and/ ...

Jupyter/Fabric Notebooks for data engineering workflows Solid understanding of data lakehouse architecture patterns and medallion architecture API Integration experience Experience working with Delta Lake or similar lakehouse storage formats Strong SQL skills for data manipulation, transformation, and quality validation Any previous experience within Manufacturing environments would ...

working with Jupyter/Fabric Notebooks for data engineering workflows Solid understanding of data lakehouse architecture patterns and medallion architecture Experience working with Delta Lake or similar lakehouse storage formats Strong SQL skills for data manipulation, transformation, and quality validation This is a role that will require ...

doing as the Machine Learning Engineer: Developing and maintaining API services using Azure and Databricks Managing caching layers with Azure Cache (Redis) Using Delta Live Tables for data processing and analytics Integrating with cloud-based data storage solutions like Snowflake Collaborating with cross-functional teams in an agile environment … from the Machine Learning Engineer: Solid experience in ML Ops, particularly with Azure and Databricks Familiarity with Postgres, Redis, and Snowflake Understanding of Delta Lake Architecture, Docker, and container services Experience building and orchestrating APIs Strong problem-solving and communication skills Bonus: exposure to Azure Functions, Containers ...

role, focused on building, optimising, and operating Databricks-based Lakehouse solutions across cloud environments. Responsibilities: Design, build, and maintain data pipelines using Databricks (Spark, Delta Lake). Develop batch and streaming workloads using PySpark/Spark SQL. Implement data ingestion, transformation, and orchestration pipelines. Optimise Databricks workloads … GCP. Skills: Strong hands-on expertise with Databricks in production environments. Advanced PySpark and Spark SQL development experience. Experience building lakehouse architectures using Delta Lake. Strong cloud experience in Azure (preferred), AWS, or GCP. Experience with orchestration tools (ADF, Airflow, dbt, or similar). Certifications – Databricks Certified Data Engineer ...

reusable pipeline patterns Orchestrating workflows in Airflow (raw > bronze > silver > gold) Implementing manifest-driven transformation using JSON configuration stored in MongoDB or CosmosDB Maintaining Delta Lake tables and trouble shoot pipeline reliability/performance Your skills and experiences: Deep understanding of modern data engineering principles , data lifecycle ...

working on behalf of a major client in the energy sector. The position focuses on designing, developing, and maintaining Azure-based data platforms-particularly Delta Lake and modern analytics services-while providing architectural leadership and hands-on click apply for full job details ...