Senior Data Engineer
- Hiring Organisation
- Hays
- Location
- Cardiff, South Glamorgan, Wales, United Kingdom
- Employment Type
- Contract
- Contract Rate
- Up to £500.0 per day + £500 Per Day
Azure Databricks using PySpark and SQL Build incremental processing using Delta Lake and Change Data Feed Extend and maintain watermark-driven CDC frameworks Implement and maintain Slowly Changing Dimension (SCD Type 2) logic Engineer time-aware joins and dimensional transformations Support optimisation of Spark workloads and Delta tables … Solid understanding of dimensional modelling and star schema Medallion Architecture Delivery Implement ingestion patterns into Bronze Build Silver state, CDC and SCD2 tables Develop Gold dimensional models aligned to Kimball principles. Maintain clean separation of Bronze, Silver and Gold usage and responsibilities Delta Lake & Performance Optimisation Work extensively with Delta ...