Senior Data Engineer
12 Month Fixed Term Contract
The Team:
The Data Engineering team is positioned in the Delivery business unit of the IT function within the Data & Automation department sitting alongside the Report Engineering, Data Governance, Data Migrations and Automation teams.
Data Engineering is a perfect blend of Data Engineers and QA Engineers that are renowned for their forward thinking and trail blazing. Exceptionally talented, supportive and personable, it’s an area that’s challenging, provides excellent growth opportunities and rewarding.
Role Expectations:
- Leading by example, through hands-on development and review of a quality Python and SQL codebase.
- Inputting into planning and estimating work / delivery roadmap.
- Keeping up to date with Databricks platform developments and maintaining tech currency of internal systems.
- Coaching and mentoring a distributed team of data engineers across multiple delivery projects.
- Managing relationships with stakeholders across operations, technology, and business teams to understand requirements and ensure.
Must Have:
- 8+ years’ experience in data engineering and 1+ experience as Lead Data Engineer
- Proven expertise with Microsoft Azure
- Strong proficiency in Databricks with hands-on Scala (Spark) and PySpark .
- Expertise in Databricks for large-scale data engineering and analytics workloads. Design and implement scalable data architectures using Databricks Unity Catalog, Delta Lake, and Apache Spark
- Implement advanced Delta Lake features including time travel, vacuum operations, and Z-ordering.
- Desirable: certification in Databricks Data Engineer and/or Azure Data Engineer are a plus.