2 of 2 Contract Delta Lake Jobs in London

develop, and maintain ETL/ELT pipelines using Airflow for orchestration and scheduling. Build and manage data transformation workflows in DBT running on Databricks . Optimize data models in Delta Lake for performance, scalability, and cost efficiency. Collaborate with analytics, BI, and data science teams to deliver clean, reliable datasets. Implement data quality checks (dbt tests, monitoring) and … Data Engineer (or similar). Hands-on expertise with: DBT (dbt-core, dbt-databricks adapter, testing & documentation). Apache Airflow (DAG design, operators, scheduling, dependencies). Databricks (Spark, SQL, Delta Lake, job clusters, SQL Warehouses). Strong SQL skills and understanding of data modeling (Kimball, Data Vault, or similar) . Proficiency in Python for scripting and pipeline development. More ❯

per day, 100% remote working and initial contract until end of 2025. Key Duties and Responsibilities Develop and optimise PySpark batch pipelines that process Parquet data and use Delta Lake for all IO, applying validation, enrichment, and billing calculation logic directly in PySpark code. Build reliable PySpark jobs that read/write Delta tables on ADLS Gen2. … with orchestrators (ADF or Container App Orchestrator) and CI/CD pipelines (GitHub Actions) Operate securely within private-network Azure environments (Managed Identity, RBAC, Private Endpoints). PySpark with Delta Lake (structured APIs, MERGE, schema evolution). Solid knowledge of Azure Synapse Spark pools or Databricks, ADLS Gen2, and Azure SQL. Familiarity with ADF orchestration and containerised Spark More ❯