5 of 5 Delta Lake Jobs in the North West

bronze, silver, and gold layers. It also implements manifest‐driven transformations using JSON configurations stored in MongoDB or CosmosDB. Responsibilities include maintaining and optimising Delta Lake tables while troubleshooting reliability and performance issues to ensure resilient, efficient, and well‐governed data pipelines. What you’ll be doing: Building … reusable pipeline patterns Orchestrating workflows in Airflow (raw > bronze > silver > gold) Implementing manifest-driven transformation using JSON configuration stored in MongoDB or CosmosDB Maintaining Delta Lake tables and trouble shoot pipeline reliability/performance Your skills and experiences: You will have deep understanding of modern data engineering principles ...

tooling Key skills Strong experience with Microsoft Fabric (Data Factory, Lakehouse, OneLake, Pipelines, SQL Warehouse) Solid data engineering background using SQL, Python, PySpark, Delta Lake E xperience building and optimising end-to-end data pipelines CI/CD experience for analytics and data platforms (Azure DevOps or GitHub ...

also contribute to best practice implementation and continuous improvement within cross-functional engineering teams. What You’ll Do Design and develop robust pipelines using DeltaLake, Spark Structured Streaming, and Unity Catalog Build real-time event-driven solutions with tools such as Kafka and Azure Event Hubs Apply DevOps principles ...

across analytical and operational datasets. Proven experience delivering complex end-to-end data platforms in Azure or Fabric environments. Strong understanding of medallion architectures, Delta Lake, distributed compute and modern data engineering principles. Experience defining data engineering standards, governance and security frameworks. Ability to engage confidently with business ...

deployments. What You’ll Do Lead the end-to-end delivery of Databricks-based solutions across Azure environments Architect secure and scalable pipelines using DeltaLake, Spark Structured Streaming, and Unity Catalog Develop and enforce engineering best practices across the data lifecycle Engage with clients to define solution strategies and data … Mentor engineering teams and contribute to internal capability development initiatives What We’re Looking For Proven experience with the Databricks platform, including Unity Catalog, DeltaLake, and orchestration Expertise in Python, PySpark, and distributed data processing frameworks Extensive background in CI/CD pipeline development with tools like Azure DevOps, Jenkins ...