1 to 25 of 30 Delta Lake Jobs in the UK

reliable delivery of new data pipelines alongside ongoing operational stability.This is a hands-on engineering role focused on implementation, optimisation and support of Delta Lake pipelines across Bronze, Silver, Staging and Gold layers, operating within an established architectural framework. Key Responsibilities Develop and maintain data pipelines in Azure … Databricks using PySpark and SQL Build incremental processing using Delta Lake and Change Data Feed Extend and maintain watermark-driven CDC frameworks Implement and maintain Slowly Changing Dimension (SCD Type 2) logic Engineer time-aware joins and dimensional transformations Support optimisation of Spark workloads and Delta tables ...

high-performance data solutions using Databricks and modern cloud technologies. Key Responsibilities Design, develop, and maintain scalable data pipelines using Databricks, Apache Spark, and Delta Lake Build and optimize ETL/ELT workflows for batch and streaming data Develop data models to support analytics, reporting, and machine learning … years of experience in data engineering or a similar role Strong experience with Databricks and Apache Spark Proficiency in Python and SQL Experience with Delta Lake, data modeling, and performance tuning Hands-on experience with cloud platforms (AWS, Azure, or GCP) Familiarity with data orchestration tools (e.g., Airflow ...

data becomes a true competitive advantage. Why consider this role? Lead delivery of a strategic cloud data platform Own architecture across Azure, Databricks, Synapse & Delta Lake Influence enterprise-wide data governance and standards Hybrid working - 3 days per week in the office Competitive salary, bonus & strong benefits High … regulated finance environment What we're looking for Proven experience as a Lead/Principal Data Engineer Deep Azure data stack knowledge (Databricks, Synapse, Delta Lake, Unity Catalog) Strong SQL and data-modelling capability Experience with orchestration, monitoring and data quality tooling Background working in regulated financial services ...

modern cloud data platform. Key Responsibilities Design and implement scalable data models within Databricks. Transform raw datasets into curated, analytics-ready data products using Delta Lake and Spark. Build and maintain semantic layers for business reporting. Translate stakeholder requirements into technical specifications. Define KPIs and controlled vocabulary across … governance standards. Support CI/CD processes and version control workflows. Required Experience Strong SQL expertise (data modelling & optimisation). Hands-on Databricks experience (Delta Lake + Spark). Experience with dbt or similar transformation frameworks. Python for data manipulation and automation. Azure cloud experience. Familiarity with ...

data models, transformation logic, and quality frameworks across the data lifecycle. * Write clean, maintainable Python code for data engineering and automation tasks. * Work with Delta Lake and modern lakehouse patterns to optimise data storage and performance. * Contribute to CI/CD and DataOps practices to improve deployment reliability … years working in data engineering, data consulting, or a similar technical role. Nice to Have Skills * Exposure to Microsoft Fabric. * Experience with Delta Lake and lakehouse architectures. * Familiarity with CI/CD, DataOps, or modern deployment pipelines. Soft Skills * Confident in client facing interactions, including workshops, requirements gathering ...

forward-thinking, data-driven organisation. If you thrive bridging the gap between data engineering and analytics, and enjoy working with tools like Databricks, Delta Lake, Spark, and dbt, this is an exciting short-term opportunity. What You'll Do: Transform raw data into structured, reliable datasets … workflows, SQL queries, and Databricks pipelines for performance. What We're Looking For: Strong SQL and data modelling skills. Hands-on experience with Databricks, Delta Lake, Spark, and preferably dbt. Python for data manipulation and automation. Familiarity with BI tools (Power BI) and cloud platforms (Azure). Understanding ...

quality, observability, and cost-efficient performance. Responsibilities • Design and implement complex ETL/ELT pipelines using Databricks (Python, SQL, DLT). • Build and optimise Delta Lake tables with effective partitioning and performance strategies. • Contribute to Lakehouse architecture design, ingestion patterns, and data product boundaries. • Ensure solutions align with … What You'll Need to Be Successful • Strong experience as a Senior Data Engineer in modern data environments. • Deep expertise in Databricks, Spark, Delta Lake, and Lakehouse architectures. • Proven ability to design scalable, modular, and maintainable data solutions. • Experience working in cloud platforms (Azure preferred). • Solid understanding ...

quality, observability, and cost-efficient performance. Responsibilities Design and implement complex ETL/ELT pipelines using Databricks (Python, SQL, DLT). Build and optimise Delta Lake tables with effective partitioning and performance strategies. Contribute to Lakehouse architecture design, ingestion patterns, and data product boundaries. Ensure solutions align with … What You'll Need to Be Successful Strong experience as a Senior Data Engineer in modern data environments. Deep expertise in Databricks, Spark, Delta Lake, and Lakehouse architectures. Proven ability to design scalable, modular, and maintainable data solutions. Experience working in cloud platforms (Azure preferred). Solid understanding ...

with architectural ownership. Key responsibilities Design and architect a secure, scalable AWS-based data platform using Databricks Build and operate pipelines using Databricks, PySpark, Delta Lake, and CI/CD Assess the existing data landscape and define migration approaches Lead early data migrations from legacy platforms into … communication and documentation are essential. What good looks like Significant hands-on experience with Databricks (engineering and architecture) Strong practical skills in: Databricks PySpark Delta Lake Data modelling and performance optimisation CI/CD pipelines Comfortable operating in a complex, programme-driven environment Able to clearly explain technical ...

work cross-functionally with stakeholders, engineers, and analysts. Technical Stack: Languages: Python (primary), SQL, Bash Cloud: Azure, AWS Tools: Airflow, DBT, Docker Data: Snowflake, Delta Lake, Redis, Azure Data Lake Infra & Ops: Terraform, GitHub Actions, Azure DevOps, Azure Monitor Desirable skills/knowledge/experience: Knowledge … data platforms (e.g. Snowflake, Azure Data Lake). Experience deploying models as APIs (FastAPI, Azure Functions). Understanding of monitoring, model performance tracking, and observability best practices. Familiarity with orchestration tools like Airflow or Azure Data Factory. ...

with excellent communication skills. Technical Stack: Languages: Python (primary), SQL, Bash Cloud: Azure, AWS Tools: MLflow, Azure ML, Airflow, Docker, Fargate, FastAPI Data: Snowflake, Delta Lake, Redis, Azure Data Lake Infra & Ops: Terraform, GitHub Actions, Azure DevOps, Grafana, Azure Monitor Desirable skills/knowledge/experience: Knowledge … data platforms (e.g. Snowflake, Azure Data Lake). Experience deploying models as APIs (FastAPI, Azure Functions). Understanding of monitoring, model performance tracking, and observability best practices. ...

help modernise legacy data estates and embed strong data engineering standards. Key Responsibilities: Build and operate scalable data pipelines using Microsoft Fabric (OneLake/Delta Lake, Data Factory, Synapse Data Engineering). Develop batch processing solutions using PySpark, Spark SQL, Python, and SQL , with a focus on performance ...

line with the organisation’s norms and values Essential • Proven experience in managing data operations, integrations, and engineering teams. • Strong expertise in Databricks, including Delta Lake, Spark, and automated pipeline development. • Hands-on experience with ETL/ELT processes, data governance, and cloud-based data platforms. • Proficiency ...

business. Contribute to architectural discussions and the ongoing data platform roadmap. Tech You'll Use Databricks: Lakeflow Declarative Pipelines, Lakeflow Workflows, Unity Catalog, Delta Lake Azure: ADLS Gen2, Data Factory, Event Hubs (optional), Key Vault, private endpoints Languages: PySpark, Spark SQL, Python, Git DevOps: Azure DevOps Repos & Pipelines … Azure + Databricks . Strong PySpark and Spark SQL skills within distributed compute environments. Experience working in a Lakehouse/Medallion architecture with Delta Lake. Understanding of dimensional modelling (Kimball), including SCD Type 1/2. Exposure to operational concepts such as monitoring, retries, idempotency and backfills. Mindset Good ...

pipelines using Azure Data Factory and Databricks. Develop scalable data models and transformations using SQL and Python . Work hands‐on with Databricks (Lakehouse, Delta tables, notebooks, workflows). Deliver high-quality dashboards and reporting solutions using Power BI . Implement best practices for data quality, governance, lineage … cost optimisation and reliability improvements across the data estate. Document pipelines, models and processes to ensure smooth knowledge transfer. Technical Skills Required Databricks – notebooks, Delta Lake, Spark (PySpark desirable) Power BI – data modelling, DAX, dashboard/report development SQL – advanced querying, performance optimisation, data modelling Python – scripting, transformation ...

improvement across the data engineering function. Responsibilities Build and maintain scalable ETL/ELT pipelines using Databricks (SQL, Python, Spark). Develop and optimise Delta Lake tables across medallion layers. Implement best practices for performance, modularity, and reusability. Support CI/CD processes for notebooks, jobs, and data ...

improvement across the data engineering function. Responsibilities Build and maintain scalable ETL/ELT pipelines using Databricks (SQL, Python, Spark). Develop and optimise Delta Lake tables across medallion layers. Implement best practices for performance, modularity, and reusability. Support CI/CD processes for notebooks, jobs, and data ...

cloud data strategy, define engineering best practice and guide delivery across cross-functional teams. Responsibilities Build and optimise Azure data platforms using ADF, Data Lake, Azure Functions and Databricks Deliver reliable, high-quality data pipelines Work with stakeholders to ensure solutions meet Lloyd's of London regulatory, audit … Data Services and Databricks in enterprise environments. Proven delivery experience within the Lloyd's of London or London Market insurance sector Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns and data modelling. Strong Python, PySpark and Spark SQL capability Solid understanding of data governance Proven track record ...

across the organisation. You'll design and develop scalable data pipelines within a modern Azure ecosystem, working with technologies such as Azure Synapse, Databricks, Delta Lake, Unity Catalog, and Databricks SQL. Responsibilities include overseeing pipeline orchestration, implementing data quality controls, monitoring and alerting, and managing data lineage, glossaries ...

wont just be delivering features; youll be setting direction. The tech youll work with Core:Azure, Databricks, Python, SQL, dbt, MQTT Storage & Serving:Delta Lake, Postgres, TimescaleDB Modelling & ML:MLflow Visualisation:Grafana What makes you a great fit Youre someone who can zoom out to architecture and zoom ...

working with Jupyter/Fabric Notebooks for data engineering workflows Solid understanding of data lakehouse architecture patterns and medallion architecture Experience working with Delta Lake or similar lakehouse storage formats Strong SQL skills for data manipulation, transformation, and quality validation This is a role that will require ...

consumption layers Technology Leadership Evaluate and recommend modern data technologies and tooling Stay current with industry trends such as open table formats (Apache Iceberg, Delta Lake), data observability, and cloud-native services Lead proofs-of-concept and technical assessments for new technologies Ensure the data architecture remains modern ...

consumption layers Technology Leadership Evaluate and recommend modern data technologies and tooling Stay current with industry trends such as open table formats (Apache Iceberg, Delta Lake), data observability, and cloud-native services Lead proofs-of-concept and technical assessments for new technologies Ensure the data architecture remains modern ...

streamlined data analysis, product builds and reduced time to market Competencies and Behaviours 1 -3 years analytics/data science experience Practical knowledge of; Delta Lake architecture and versioned datasets, Data pipelines, orchestration and scheduling concepts Proficiency in analytical programming language such as python and/ ...

components to the position: Define the long-term technical vision and roadmap for our modern Data Platform, Mobius (including Data Warehousing, Data Lake, Streaming, and Governance tooling). Design and govern the reference architecture for core data infrastructure, ensuring optimal scalability, reliability, and security. Act as a hands … software and data engineering, with provable experience operating at a Principal, Staff, or equivalent level. Deep proficiency across the modern data stack (Snowflake, BigQuery, Delta Lake, Iceberg, Kafka, Flink). Proven track record designing scalable, self-service data platforms using cloud-native services (AWS/GCP) and infrastructure ...