Senior Data Engineer - £62,000 - Hybrid

SENIOR DATA ENGINEER£62,000 - Hybrid Working (2 days a week in office) - Bridgend, Wales.Your new company The company is a not-for-profit housing association established in 2003 as the first large-scale voluntary transfer of council-owned homes to a social housing provider in Wales. The organisation is rooted in its mission to provide safe, high-quality and affordable homes across Bridgend and South Wales, creating thriving communities where people feel secure, supported, and proud to live.

Your new role The overall purpose of this role is to be the primary architect and builder of our modern data estate. This is a high impact, "greenfield" role where you will design and implement the Azure Databricks infrastructure that forms the backbone of our future data, BI, and AI capabilities, and continue to mature our PowerBI architecture for ease and robustness of reporting and self-service.This role will champion a DataSecOps approach, ensuring that security, privacy and compliance are embedded throughout the data lifecycle from the outset.The nascent nature of our data capabilities means you will have the opportunity to help shape those capabilities to achieve lasting positive value for the organisation as a whole, our customers and our colleagues.Key responsibilities and accountabilitiesData Platform Architecture & DataSecOps- Lead the design and implementation of a scalable and secure data platform using Azure Databricks, enabling the ingestion, transformation and governance of data from key business systems while supporting a reliable "single version of the truth" for reporting and analytics;- Design and deploy secure, scalable ELT or ETL pipelines using Databricks (Unity Catalog), Delta Lake, and Azure Data Lake Storage (ADLS);- Implement shift-left security principles within the data ingestion layer, including automated data masking, encryption at rest and in transit, and row- and column-level security controls;- Establish and embed DataOps and DataSecOps practices, including CI/CD, version control (Git), and automated security scanning of data pipelines.Power BI & Reporting Foundation- Finalise a governed Power BI architecture, including standardised semantic models and dataflows;- Ensure robust Identity and Access Management (IAM) across Power BI workspaces, enabling true self-service analytics without compromising data privacy.Data Stewardship & Leadership- Proactively identify data and logic inconsistencies between systems, providing clear evidence and insight to support stakeholders in resolving operational issues;- Act as a technical lead, coaching a Junior Data Engineer and Analyst on modern engineering standards and secure coding practices;- Work with the Software Engineering Manager to establish naming conventions, metadata standards, and data residency/sovereignty protocols.Stakeholder Engagement & Continuous Improvement- Constructively challenge stakeholders and teams where necessary on scope and delivery to ensure deliverables remain on track;- Support continuous improvement efforts through ongoing analysis and feedback, and implementation of best practices;- Work with housing and frontline teams to ensure system implementations improve service quality and accessibility for customers;What you'll need to succeed Knowledge/Qualifications - Expert-level SQL and Python (ideally PySpark).- A strong understanding of DataSecOps; experience with PII handling, data obfuscation, and RBAC (Role-Based Access Control).- Proven ability to design data systems (preferably Medallion architectures).- Advanced knowledge of Power BI security and administration.- Familiarity with git and gitflow version control processes;- Knowledge of cloud cost and resource optimisation (FinOps principles)- Awareness of agile delivery methodologies (Scrum or Kanban).Desirable:- Experience with DBT (Data Build Tool) or Prefect.- Awareness of AI/ML data preparation and the security implications of LLM training data.- Knowledge of housing, asset management, or repairs systems (ideally Civica Cx and Total Mobile Service Connect).Experience- Implemented and operated cloud-based data platforms, ideally on Azure (experience MUST include build, not just maintenance of existing).- Delivered a Delta Lakehouse, Medallion Architecture, or similar curated data layering approach.- Created end-to-end ELT/ETL pipelines using Databricks, Spark, Azure DataFactory, or equivalent tooling.- Embedded security controls into pipelines (e.g., secure credential handling, encryption, RBAC, ACLs).- Worked with CI/CD, version control (Git), code review processes, and automated testing for data.- Designed data models (star schema, transformed delta tables, data marts) for analytics and BI.- Ensured data quality through validation, profiling, reconciliation, and automated tests.- Improved or restructured existing reporting environments to make them scalable, governed, and self-service friendly.- Provided guidance, reviewed code, coached junior team members, and influenced team practices.Desirable:- Design of Data architecture, as well as build.- Azure Databricks experience- Experience managing multiple workstreams simultaneously, balancing major projects with smaller initiatives.

What you need to do now If you're interested in this role, click 'apply now' to forward an up-to-date copy of your CV.

Hays Specialist Recruitment Limited acts as an employment agency for permanent recruitment and employment business for the supply of temporary workers. By applying for this job you accept the T&C's, Privacy Policy and Disclaimers which can be found at hays.co.uk