Data Engineer

The Opportunity

A specialist data and AI consultancy is growing its Databricks capability and is looking for a Senior Databricks Consultant to join its delivery team.

This is an opportunity to work on modern cloud data platforms and Databricks Lakehouse programmes, helping enterprise clients design, build and optimise scalable data engineering solutions.

You’ll work closely with experienced data architects, engineers and client stakeholders, with Databricks at the centre of the role. The position would suit someone who enjoys hands-on engineering while also contributing to technical solution design and client delivery.

What You'll Be Doing

• Design, develop and maintain data pipelines and data engineering solutions using Databrick
• sBuild scalable data processing solutions using PySpark and SQL
• Work across Databricks Lakehouse environments, supporting implementation, optimisation and ongoing improvement
• Develop and optimise batch and data processing workflows
• Contribute to technical solution design and data platform architecture
• Work closely with technical and business stakeholders to understand requirements and deliver effective solutions
• Support testing, deployment and production implementation activities
• Troubleshoot performance, reliability and data quality issues
• Follow and contribute to engineering best practices across Databricks and cloud data platforms
• Support clients through different stages of their data platform and lakehouse transformation journeys

What We're Looking For

• Strong commercial experience in Data Engineering
• Hands-on experience with Databricks and/or Apache Spark
• Strong PySpark and SQL development skills
• Experience designing, building and supporting data pipelines
• Experience working with Azure or AWS
• Good understanding of modern data platforms and Lakehouse architectures
• Strong analytical and problem-solving abilities
• Excellent communication skills, with the ability to work effectively with both technical and non-technical stakeholders

Desirable Experience

Experience with any of the following would be highly beneficial

• Delta Lake
• Unity Catalog
• MLfow
• Databricks optimisation and performance tuning
• AI, Machine Learning or Generative AI initiatives
• AI-assisted development tools such as Claude, GitHub Copilot or Cursor
• Consultancy, professional services or client-facing delivery experience
• Active or previous SC Clearance