10 of 10 Contract Delta Lake Jobs in London

data into structured, analytics-ready formats. Optimise pipeline performance and ensure high availability of data services. Infrastructure & Architecture Architect and deploy scalable data lake solutions using Azure Data Lake Storage. Implement governance and security measures across the platform. Leverage Terraform or similar IaC tools for controlled and reproducible … deployments. Databricks Development Develop and optimise data jobs using PySpark or Scala within Databricks. Implement the medallion architecture (bronze, silver, gold layers) and use Delta Lake for reliable data transactions. Manage cluster configurations and CI/CD pipelines for Databricks deployments. Monitoring & Operations Implement monitoring solutions using Azure ...

modern cloud data platform. Key Responsibilities Design and implement scalable data models within Databricks. Transform raw datasets into curated, analytics-ready data products using Delta Lake and Spark. Build and maintain semantic layers for business reporting. Translate stakeholder requirements into technical specifications. Define KPIs and controlled vocabulary across … governance standards. Support CI/CD processes and version control workflows. Required Experience Strong SQL expertise (data modelling & optimisation). Hands-on Databricks experience (Delta Lake + Spark). Experience with dbt or similar transformation frameworks. Python for data manipulation and automation. Azure cloud experience. Familiarity with ...

forward-thinking, data-driven organisation. If you thrive bridging the gap between data engineering and analytics, and enjoy working with tools like Databricks, Delta Lake, Spark, and dbt, this is an exciting short-term opportunity. What You'll Do: Transform raw data into structured, reliable datasets … workflows, SQL queries, and Databricks pipelines for performance. What We're Looking For: Strong SQL and data modelling skills. Hands-on experience with Databricks, Delta Lake, Spark, and preferably dbt. Python for data manipulation and automation. Familiarity with BI tools (Power BI) and cloud platforms (Azure). Understanding ...

tests (Behave), including mocking and patching Using PySpark to process large-scale datasets while keeping business logic in Python Creating, maintaining, and optimising Delta Lake tables for performance and reliability Building and running applications in containerised (Docker) environments Integrating Python applications with Azure services such as Azure Functions … Python development experience (essential) Proven experience writing test-driven Python code in production environments Solid data engineering experience using PySpark Experience working with Delta Lake Hands-on experience with Docker and containerised workflows Good knowledge of Azure and integrating Python applications with cloud services A software-engineering mindset ...

work cross-functionally with stakeholders, engineers, and analysts. Technical Stack: Languages: Python (primary), SQL, Bash Cloud: Azure, AWS Tools: Airflow, DBT, Docker Data: Snowflake, Delta Lake, Redis, Azure Data Lake Infra & Ops: Terraform, GitHub Actions, Azure DevOps, Azure Monitor Desirable skills/knowledge/experience: Knowledge … data platforms (e.g. Snowflake, Azure Data Lake). Experience deploying models as APIs (FastAPI, Azure Functions). Understanding of monitoring, model performance tracking, and observability best practices. Familiarity with orchestration tools like Airflow or Azure Data Factory. ...

with excellent communication skills. Technical Stack: Languages: Python (primary), SQL, Bash Cloud: Azure, AWS Tools: MLflow, Azure ML, Airflow, Docker, Fargate, FastAPI Data: Snowflake, Delta Lake, Redis, Azure Data Lake Infra & Ops: Terraform, GitHub Actions, Azure DevOps, Grafana, Azure Monitor Desirable skills/knowledge/experience: Knowledge … data platforms (e.g. Snowflake, Azure Data Lake). Experience deploying models as APIs (FastAPI, Azure Functions). Understanding of monitoring, model performance tracking, and observability best practices. ...

help modernise legacy data estates and embed strong data engineering standards. Key Responsibilities: Build and operate scalable data pipelines using Microsoft Fabric (OneLake/Delta Lake, Data Factory, Synapse Data Engineering). Develop batch processing solutions using PySpark, Spark SQL, Python, and SQL , with a focus on performance ...

with SQL and Python. Background in data integration/ingestion using tools such as Informatica IICS, Azure Data Factory, notebooks and Databricks. Experience with Delta Lake, data warehousing technologies and Azure cloud services. Proven experience modelling, integrating and transforming insurance domain data, Experience with both on-prem ...

Responsibilities Define and maintain the end-to-end architecture for Azure Databricks-based data platforms. Design Lakehouse architectures using Azure Databricks, Azure Data Lake Storage (ADLS Gen2), and associated Azure services. Produce and maintain High-Level Designs (HLDs) and Low-Level Designs (LLDs). Establish and govern data ingestion … dependencies and support mitigation planning. Essential Skills and Experience Demonstrable experience designing Azure Databricks architectures in enterprise environments. Strong understanding of Databricks Lakehouse architecture, Delta Lake, and governance concepts. Architectural-level knowledge of Spark-based platforms. Experience designing secure, governed data platforms in regulated or public sector environments. ...

constraints, contributing to mitigation plans. Essential Skills & Experience Strong track record designing Azure Databricks-based architectures in enterprise environments. Deep understanding of Lakehouse principles, Delta Lake , and governance capabilities. Architectural expertise with Spark-based platforms. Experience designing secure, governed platforms within regulated or public sector contexts. Proven ability ...