Databricks Tech Lead - Azure Data Engineering (Contract)

Databricks Tech Lead - Azure Data Engineering (Contract)

We are looking for a highly technical, hands-on Databricks Tech Lead to join a large-scale digital transformation programme within the Energy Trading space.

This role will focus on building, optimising, and scaling a data lake on Azure with particular emphasis on streaming pipelines, Delta Live Tables (DLT), and cost-efficient architecture.

Role:

• Designing and developing streaming and batch data pipelines in Databricks (including DLT).
• Managing structured streaming in continuous mode, ensuring low-latency data delivery.
• Collaborating with architects, consultants, and business stakeholders to build a unified Azure-based lakehouse.
• Troubleshooting and optimising performance bottlenecks in high-volume streaming systems.
• Applying best practices for cloud cost management and efficient Databricks resource usage.
• Integrating data from on-prem and cloud sources using tools like Qlik Replicate (CDC).
• Supporting future phases involving data science and machine learning enablement.

Tech Environment

• Platform: Azure
• Core Tools: Databricks, Delta Live Tables, Qlik Replicate (CDC)
• Architecture: Hybrid (on-prem + cloud)
• Data Modes: Real-time streaming & batch jobs

If you are a Tech Lead with a strong background in Databricks, Delta Live Tables and Qlik Replicate, please apply with your updated CV.