Databricks Architect

Role summary

This role focuses on designing and delivering a cloud-based data platform using Databricks on AWS, as part of a wider transformation programme spanning systems, processes, and ways of working.

You’ll be responsible for shaping the platform architecture, migrating legacy data into the new environment, and ensuring the platform is scalable, performant, and trusted. The role sits within a fast-paced, programme-led environment and combines hands-on engineering with architectural ownership.

Key responsibilities

• Design and architect a secure, scalable AWS-based data platform using Databricks

• Build and operate pipelines using Databricks, PySpark, Delta Lake, and CI/CD

• Assess the existing data landscape and define migration approaches

• Lead early data migrations from legacy platforms into the new environment

• Define ingestion, processing, storage, validation, and access patterns

• Implement robust data validation and verification processes

• Identify opportunities to simplify, standardise, and consolidate data during migration

• Work closely with technical and non-technical stakeholders, clearly documenting decisions

This is not a pure build role — strong communication and documentation are essential.

What good looks like

• Significant hands-on experience with Databricks (engineering and architecture)

• Strong practical skills in:

  • Databricks

  • PySpark

  • Delta Lake

  • Data modelling and performance optimisation

  • CI/CD pipelines

• Comfortable operating in a complex, programme-driven environment

• Able to clearly explain technical decisions to mixed audiences

 

Please apply with your latest CV and we'll be in touch!