Senior Data Engineer

Senior Data Engineer - Azure Databricks

Location: Hybrid (1-2 days a week in London)

Initial 6 months contract

Overview

An established technology organisation that is now going through significant change and growth is looking for a Senior Data Engineer to help design, build, and secure a modern data platform that supports business-critical systems in regulated environments. This is a really exciting greenfield project to establish a new data platform at speed.

The Role

You'll take ownership of the end-to-end data platform, including pipelines, governance, and security. Working closely with engineers, analysts, and stakeholders, you'll establish scalable, repeatable data practices and help shape the future data architecture.

Key Responsibilities

• Design and build end-to-end data pipelines, data lakes, and data warehouses using Azure Databricks
• Integrate ETL/ELT tools and cloud services for batch and real-time data processing
• Experience building data pipelines on-prem
• Define and implement standards for data modelling, orchestration, version control, and data quality (e.g. dbt, Airflow)
• Embed security best practices, including RBAC, encryption, masking, and regulatory compliance
• Collaborate with technical and non-technical stakeholders to define data requirements, SLAs, and governance frameworks
• Optimise data platform performance, scalability, and cost efficiency
• Automate monitoring, alerting, and recovery to ensure reliability and data integrity
• Drive adoption of DevOps practices for data, including CI/CD and infrastructure as code

About You

You'll be someone who:

• Has a security-first mindset and strong understanding of responsible data handling
• Is comfortable working in ambiguous environments and owning delivery end to end
• Thrives in fast-paced, collaborative teams
• Is proactive about continuous improvement and learning new data technologies
• Communicates clearly and shares knowledge with others

Experience & Skills

• Strong experience as a Data Engineer working with modern data platforms
• Hands-on expertise with Azure Databricks
• Experience building data pipelines on-prem
• Proven background building and supporting production-grade data pipelines
• Solid understanding of data governance, security, and compliance
• Experience optimising distributed data systems for performance and cost