Databricks Solution Architect

Consultancy client are looking to recruit a Databricks Solution Architect to define and deliver enterprise-scale data platform architectures using Azure Databricks.

Candidates must possess deep hands-on Databricks engineering knowledge alongside experience designing secure, scalable and highly available data platforms for UK Public Sector organisations. This is a technical architecture role requiring the ability to move between architecture design, technical implementation and engineering leadership.

Essential Technical Skills

  • 5+ years of hands-on Databricks platform experience.
  • Extensive Azure Databricks architecture experience.
  • Expert-level PySpark, Spark SQL and Python knowledge.
  • Deep understanding of Apache Spark processing architecture.
  • Extensive Delta Lake and Lakehouse architecture experience.
  • Strong experience designing Medallion Architecture.
  • Expert knowledge of Unity Catalog, governance, permissions and data lineage.
  • Experience designing Databricks workspaces, clusters, compute policies and networking.
  • Knowledge of private endpoints, managed identities, Key Vault and secure connectivity.
  • Experience with Azure Data Lake Storage Gen2, Azure Data Factory and Azure services.

Public Sector Experience

  • Extensive experience delivering enterprise data architectures
  • Understanding of UK Government security principles and data governance requirements.
  • Experience architecting platforms handling sensitive and restricted government data.
  • Knowledge of UK GDPR, Data Protection Act 2018 and relevant government security standards.
  • Experience producing architecture documentation, HLDs, LLDs and technical design authorities.
  • Experience working with public sector enterprise architecture and governance boards.

Security Clearance Requirements

  • Be eligible and willing to undergo SC clearance
  • Previous experience working on sensitive government projects is advantageous.

Responsibilities

  • Own the end-to-end Databricks solution architecture.
  • Define scalable and secure Lakehouse architecture.
  • Establish engineering and architecture standards.
  • Design data ingestion, transformation and consumption patterns.
  • Define Unity Catalog and enterprise data governance models.
  • Provide technical direction to data engineering teams.

Job Details

Company
GIS Recruiter Ltd
Location
London, United Kingdom
Employment Type
Permanent
Salary
£90000 - £120000/annum Hybrid Working
Posted