Enterprise Databricks Data Architect

Location: London (Hybrid, 1-2 days per week)Duration: Initial 6 months+

Robert Half are supporting a leading global risk consulting firm organisation on a major Finance Data Lake, Self-Service Reporting and AI Enablement programme and are seeking an experienced Enterprise Data Architect with deep Databricks expertise. This is a hands-on architecture role requiring a strong blend of Enterprise Data Architecture, Data Modelling, Databricks Engineering, Azure Data Platform expertise and Dynamics 365 F&O integration experience. The successful candidate will be responsible for defining enterprise standards, implementing scalable data frameworks and ensuring a robust foundation for future AI-driven reporting and analytics.

• Define and own Enterprise Data Architecture principles, standards and best practices
• Design scalable Databricks Lakehouse solutions using Medallion Architecture (Bronze, Silver, Gold)
• Create enterprise-wide data models supporting Finance, Analytics and Reporting use cases
• Design and implement reusable Data Engineering Frameworks and Solution Accelerators
• Develop metadata-driven ingestion, transformation and orchestration frameworks
• Define governance, lineage, security and cataloguing standards across the platform
• Provide architecture leadership across Databricks, Azure and Power BI ecosystems
• Work closely with Finance, Data Engineering, Product and Business stakeholders
• Support AI and Agentic AI enablement by establishing trusted and governed data foundations
• Mentor engineering teams and promote engineering best practices

• Strong Enterprise Data Architecture background
• Experience designing and implementing Enterprise Data Frameworks, Standards and Architecture Patterns
• Expert Data Modelling skills (Conceptual, Logical, Physical and Dimensional)
• Extensive Azure Databricks experience within large-scale enterprise environments
• Databricks Lakehouse Architecture and Medallion Design Patterns
• Databricks Data Engineering using PySpark and Spark SQL
• Delta Live Tables (DLT)
• Unity Catalog implementation and governance
• Databricks Workflows
• Databricks Asset Bundles (DABs)
• Databricks Solution Accelerators
• Metadata-Driven Ingestion and Orchestration Frameworks
• MLflow, Feature Store and MLOps Frameworks
• Azure Data Platform (ADLS, Azure Data Factory, Synapse, Databricks)
• Azure DevOps, CI/CD and Infrastructure as Code
• Enterprise Data Governance, Data Quality and Data Lineage Frameworks
• Experience integrating Dynamics 365 Finance & Operations (D365 F&O) into enterprise data platforms
• Experience delivering Finance Data Lake, Data Warehouse or Lakehouse programmes
• Strong stakeholder engagement and solution design capabilities

• Previous experience working within specialist Databricks consultancies or Databricks partner organisations
• Mosaic AI and AI-enabled Data Platform delivery
• Agentic AI / GenAI implementation within Databricks
• Microsoft Fabric
• Financial Services, Real Estate, Insurance or Retail sector experience
• Experience leading Data Architecture workstreams within large transformation programmes
• Experience implementing self-service reporting and analytics platforms
• Experience supporting Power BI semantic models and enterprise reporting environments

• Azure Databricks
• Delta Live Tables (DLT)
• Unity Catalog
• Databricks Asset Bundles (DABs)
• Databricks Workflows
• PySpark
• Spark SQL
• MLflow
• Mosaic AI
• Azure Data Factory
• Azure Synapse
• Azure Data Lake Storage
• Dynamics 365 Finance & Operations (D365 F&O)
• Azure DevOps
• Power BI
• Terraform
• CI/CD

We are particularly interested in Enterprise Data Architects who have built and implemented Databricks frameworks, metadata-driven architectures, reusable engineering patterns and enterprise-scale data platforms, rather than candidates with only high-level Databricks exposure. Experience working on large-scale Azure and Databricks transformation programmes, ideally involving Finance data and D365 F&O integrations, is highly desirable.

Robert Half Ltd acts as an employment business for temporary positions and an employment agency for permanent positions. Robert Half is committed to diversity, equity and inclusion. Suitable candidates with equivalent qualifications and more or less experience can apply. Rates of pay and salary ranges are dependent upon your experience, qualifications and training. If you wish to apply, please read our Privacy Notice describing how we may process, disclose and store your personal data: