2 of 2 Remote Contract Delta Lake Jobs

remote working and initial contract until end of 2025, must be CTC cleared. Key Duties and Responsibilities Develop and optimise PySpark batch pipelines that process Parquet data and use Delta Lake for all IO, applying validation, enrichment, and billing calculation logic directly in PySpark code. Build reliable PySpark jobs that read/write Delta tables on ADLS … with orchestrators (ADF or Container App Orchestrator) and CI/CD pipelines (GitHub Actions) Operate securely within private-network Azure environments (Managed Identity, RBAC, Private Endpoints). PySpark with Delta Lake (structured APIs, MERGE, schema evolution). Solid knowledge of Azure Synapse Spark pools or Databricks, ADLS Gen2, and Azure SQL. Familiarity with ADF orchestration and containerised Spark More ❯

engineering, with a strong focus on building scalable data pipelines Expertise in Azure Databricks, including building and managing ETL pipelines using PySpark or Scala Solid understanding of Apache Spark, Delta Lake, and distributed data processing concepts Hands-on experience with Azure Data Lake Storage, Azure Data Factory, and Azure Synapse Analytics Proficiency in SQL and Python for More ❯