Contract Databricks Data Engineer

Databricks Data Engineer (Contract) - Mid to Senior Level

Location: London (Hybrid - 2 days per week onsite)Contract Length: Initial 6 months Day Rate: Flexible (Inside & Outside IR35 considered - final determination pending)

We're partnering with a client seeking a skilled Databricks Data Engineer to support the build and evolution of a modern data platform. This is a hands-on contract role, ideally suited to someone who thrives working across both new platform development and ongoing production support within a Databricks-driven environment.

The Role

This role sits at the heart of a Databricks Lakehouse implementation, where you'll be responsible for building, optimising, and maintaining scalable data pipelines and datasets to support analytics and business intelligence.

You'll work closely with technical and business stakeholders to deliver high-quality, performant data solutions while also ensuring the stability and reliability of existing workloads.

Key Responsibilities

Design, build, and optimise data pipelines within Azure Databricks using PySpark and Spark SQL
Develop and manage Delta Lake-based data models, supporting a structured Lakehouse / Medallion architecture (Bronze, Silver, Gold layers)
Support the ingestion and transformation of large-scale data from multiple sources into Databricks
Contribute to the modernisation and migration of legacy SQL Server workloads into Databricks
Monitor, troubleshoot, and improve the performance of existing data pipelines and jobs
Work closely with stakeholders to ensure data is reliable, well-structured, and ready for analytics and reporting
Integrate and orchestrate workflows using tools such as Azure Data Factory or Databricks Workflows
Collaborate with BI teams to ensure datasets are optimised for Power BI and downstream consumption

Required Skills & Experience

Strong commercial experience working with Azure Databricks as a core data processing platform
Deep expertise in:
- PySpark / Apache Spark for distributed data processing
- Delta Lake (ACID transactions, optimisation, data versioning)
- Spark SQL and advanced SQL techniques
- Python for data engineering and pipeline development
Hands-on experience designing and implementing Lakehouse architectures
Experience migrating data platforms from on-premises systems into cloud-based Databricks environments
Solid understanding of data modelling, ETL/ELT design, and performance optimisation
Familiarity with orchestration tools such as Azure Data Factory, Airflow, or Databricks Jobs
Experience working in production environments with both project delivery and BAU responsibilities

Desirable Experience

Exposure to Unity Catalog, data governance, and fine-grained access control in Databricks
Experience implementing CI/CD pipelines for Databricks (e.g. via Azure DevOps or Git integration)
Knowledge of streaming data pipelines (Spark Structured Streaming / Kafka)
Experience working with cloud-native data architectures in Azure
Prior exposure to Power BI or similar BI/reporting tools

Please Note: This is a contract role for UK residents only. This role does not offer Sponsorship. You must have the right to work in the UK with no restrictions. Some of our roles may be subject to successful background checks including a DBS and Credit Check. TRG are the go-to recruiter for Power BI and Azure Data Platform roles in the UK, offering more opportunities across the country than any other. We're the proud sponsor and supporter of SQLBits, Power Platform World Tour, the London Power BI User Group, Newcastle Power BI User Group and Newcastle Data Platform and Cloud User Group. To find out more and speak confidentially about your job search or hiring needs, please contact me directly via email. - re write this spec for a data engineer aligned with Databricks

Apply Now

Contract Databricks Data Engineer

Job Details