19 of 19 Delta Lake Jobs in the UK excluding London

modelling for Finance and Enterprise data products, working closely with architecture where required. Implement and maintain data pipelines and ETL workflows in Databricks (PySpark, Delta Lake). Contribute to CI/CD pipelines for data applications using Azure DevOps and infrastructure-as-code (Terraform) in line with established … within the team and communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous). Experience contributing ...

modelling for Finance and Enterprise data products, working closely with architecture where required. Implement and maintain data pipelines and ETL workflows in Databricks (PySpark, Delta Lake). Contribute to CI/CD pipelines for data applications using Azure DevOps and infrastructure-as-code (Terraform) in line with established … within the team and communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous). Experience contributing ...

modelling for Finance and Enterprise data products, working closely with architecture where required. Implement and maintain data pipelines and ETL workflows in Databricks (PySpark, Delta Lake). Contribute to CI/CD pipelines for data applications using Azure DevOps and infrastructure-as-code (Terraform) in line with established … within the team and communicate clearly with technical and non-technical stakeholders. Data Engineer (New Technology/Microsoft) Strong experience with Databricks (Spark, PySpark, Delta Lake, and Unity Catalog advantageous). Proficiency in Azure data services (Azure Data Factory, Data Lake, Azure Functions advantageous). Experience contributing ...

zone into the Bronze layer, working in partnership with the Development and Application Management team.Design and optimise ETL/ELT pipeline frameworks using Databricks, Delta Lake and Azure Data Factory. * Ensure Silver and Gold layer data products are fit for purpose for analytics, pricing, AI and ML model … scalability. Experience in regulated financial services, ensuring compliance with industry standards. Deep expertise in data modelling - conceptual, logical and physical. Data warehousing and data lake architecture for high-performance analytics. ETL/ELT pipeline development and optimisation to support large-scale data processing. Data integration across structured and unstructured ...

Data Engineer will design, build, support and operate data pipelines and core data platform technologies within the organisation's Azure-based analytics and lake house environment. The role is hands-on and implementation-focused, working within established architectural patterns to deliver reliable, secure and well-governed data for analytics … Responsibilities Develop and maintain data pipelines using Azure Databricks (Python, PySpark and SQL) Implement ingestion and transformation across Bronze, Silver and Gold layers using Delta Lake Support and operate Azure Databricks and Azure Synapse platforms, including performance monitoring and configuration Build Gold-layer datasets optimised for Power ...

energy market data using Python, SQL and Spark. Support data flows ensuring generation and storage assets deliver the correct contracted schedules. Design and manage Delta Lake tables and SQL databases, applying strong data modelling and optimisation strategies. Develop monitoring dashboards and alerting to ensure transparency around pipeline health … similar production-focused role Strong Python (Pandas) and SQL skills Experience building and maintaining data pipelines Hands-on experience with Spark, Databricks or Delta Lake/Lakehouse architectures Familiarity with cloud data platforms, particularly within Azure environments Good understanding of data modelling, storage formats and performance optimisation Experience ...

bronze, silver, and gold layers. It also implements manifest‐driven transformations using JSON configurations stored in MongoDB or CosmosDB. Responsibilities include maintaining and optimising Delta Lake tables while troubleshooting reliability and performance issues to ensure resilient, efficient, and well‐governed data pipelines. What you’ll be doing: Building … reusable pipeline patterns Orchestrating workflows in Airflow (raw > bronze > silver > gold) Implementing manifest-driven transformation using JSON configuration stored in MongoDB or CosmosDB Maintaining Delta Lake tables and trouble shoot pipeline reliability/performance Your skills and experiences: You will have deep understanding of modern data engineering principles ...

framework for RC, PFE and EAD calculations, including netting sets, supervisory factors and collateral/CSA logic.* **Design a modern Databricks architecture** using Delta Lake, Unity Catalog and Spark/PySpark to support versioned, reproducible and regulator‐defensible exposure calculations.* **Develop high‐performance compute pipelines** that support intraday … Deep understanding of SA‐CCR** (RC, PFE, EAD), exposure methodology, netting sets, supervisory parameters and collateral/CSA design.* **Strong Databricks engineering capability**, including Delta Lake versioning, Unity Catalog governance, Spark/PySpark optimisation and secure data architecture.* **Experience designing governed, audit‐ready systems**, with a focus ...

Design, build, and optimize robust data pipelines in Databricks, feeding ML models and analytics workloads. Develop and maintain scalable data architectures including Azure Data Lake, Azure SQL Data Warehouse, and Synapse Analytics. Build and maintain reliable ETL/ELT workflows, ensuring high data quality, accessibility, and performance. Collaborate with … Model Management: MLflow, Model versioning, CI/CD Cloud & Big Data: Databricks, Databricks APIs, MCP, Blob Storage Data Engineering: ETL/ELT pipelines, DLT, Delta Lake Infrastructure as Code (IaC): Terraform, ARM templates, Ansible Version Control & DevOps: Git, Azure DevOps, Jenkins Experience 3+ years of experience ...

tooling Key skills Strong experience with Microsoft Fabric (Data Factory, Lakehouse, OneLake, Pipelines, SQL Warehouse) Solid data engineering background using SQL, Python, PySpark, Delta Lake E xperience building and optimising end-to-end data pipelines CI/CD experience for analytics and data platforms (Azure DevOps or GitHub ...

cloud data strategy, define engineering best practice and guide delivery across cross-functional teams. Responsibilities Build and optimise Azure data platforms using ADF, Data Lake, Azure Functions and Databricks Deliver reliable, high-quality data pipelines Work with stakeholders to ensure solutions meet Lloyd's of London regulatory, audit … Data Services and Databricks in enterprise environments. Proven delivery experience within the Lloyd's of London or London Market insurance sector Deep understanding of Delta Lake, Medallion architecture, Lakehouse patterns and data modelling. Strong Python, PySpark and Spark SQL capability Solid understanding of data governance Proven track record ...

across analytical and operational datasets. Proven experience delivering complex end-to-end data platforms in Azure or Fabric environments. Strong understanding of medallion architectures, Delta Lake, distributed compute and modern data engineering principles. Experience defining data engineering standards, governance and security frameworks. Ability to engage confidently with business ...

PySpark, Spark SQL and Python Implement and maintain CI/CD practices Enhance and optimise existing data pipelines About You Proven experience in Databricks, Delta Lake, PySpark and Spark SQL Excellent communication skills - ability to partner with the business and key stakeholders to champion best practice in data ...

multi-site business environment Strong proficiency in Python and SQL, with hands-on experience in Spark/PySpark and modern table formats (e.g. Delta Lake, Iceberg, Hudi) Expertise in AWS data services (e.g. S3, Glue, Redshift, Lambda, Step Functions, Kinesis) and infrastructure-as-code (Terraform or CDK) Experience ...

streamlined data analysis, product builds and reduced time to market Competencies and Behaviours 1 -3 years analytics/data science experience Practical knowledge of; Delta Lake architecture and versioned datasets, Data pipelines, orchestration and scheduling concepts Proficiency in analytical programming language such as python and/ ...

based development workflows Experience delivering features end-to-end in distributed teams Confident troubleshooting production issues in real time Desirable Skills Apache Spark (EMR) & Delta Lake PostgreSQL (Aurora) CI/CD using GitHub Actions IaC (Helm, CloudFormation) Monitoring tools (Sentry) Role Responsibilities You'll be hands ...

providing insight, documentation and explanation of new features and changes. Knowledge & Technical Skills Strong understanding of modern data platforms, particularly Databricks Practical knowledge of: Delta Lake architecture and versioned datasets Databricks notebooks (SQL and Python) Data pipelines, orchestration and scheduling concepts Solid SQL skills with the ability ...

. Configure and maintain CI/CD pipelines using GitHub Actions. Integrate with the Databricks data platform via Unity Catalog and Delta Lake. Troubleshoot and resolve integration issues across distributed systems. Document integration patterns, APIs, and architectural decisions. Drive and collaborate in the continued adoption of AI enhanced development ...

will have over 7 years of experience in regulatory technology within financial services, deep knowledge of SA‐CCR methodology, and strong skills in Databricks, Delta Lake, and Spark. This hybrid role offers competitive compensation, a supportive work culture, and the opportunity to innovate in a compliance-focused environment. ...