SC Cleared Data Engineer
Job Title: Data Engineer - Active SC Clearance Required
Location: Leeds, 1x per week
Duration: 3 months with strong chance of extension
Contract: Outside IR35
*Current active SC Clearance is essential. Applications from candidates who are only eligible for SC Clearance cannot be considered.*
This is a great opportunity for an experienced Data Engineer to support the delivery of a greenfield Azure cloud data platform for a major transformation programme. This contract will see you building a modern Azure data platform that brings together complex data from multiple systems, improving data quality, automation and reporting while helping create a single, trusted view of customer information.
Key Responsibilities
- Design, build and optimise Azure data pipelines using Azure Data Factory and Azure Databricks
- Develop data ingestion solutions using Change Data Capture and Managed File Transfer
- Build canonical data models, conformed dimensions and fact tables
- Deliver curated Gold-layer datasets for reporting, analytics and AI use cases
- Automate data validation and quality assurance processes, replacing manual checks
- Configure Unity Catalog, clusters, pipelines and data contracts
- Contribute to data governance, security, lineage and platform design
- Collaborate on solution design, code reviews, testing and defect resolution
Essential Skills & Experience
- Current active SC Clearance (mandatory)
- Strong commercial experience with Azure Databricks, Azure Data Factory and Azure data engineering
- Advanced Python (including Pandas) and SQL
- Hands-on experience with Unity Catalog
- Experience designing data warehouses, including slowly changing dimensions (SCD) and dimensional modelling
- Knowledge of CDC, data ingestion patterns and governed data platforms
- Understanding of data governance, security and lineage
- Experience within regulated or public sector environments is advantageous
- Familiarity with Power BI is beneficial but not essential