Senior Data Governance Engineer

This role sits within a client-side data governance implementation on AWS, with Databricks (Unity Catalog) as the core data platform. This opportunity is being resourced initially on a contract basis, with an emphasis on hands-on delivery from day one.

You'll own the definition and build of data quality rules on Databricks across the client's AWS-based data estate. This is a delivery-focused position: designing and implementing data quality rules using native Databricks capability (Unity Catalog, Lakehouse Monitoring, DQX) alongside a supporting data quality framework where needed (eg Great Expectations, Soda, or similar), with rules built to be accurate, tested, and ready for production.

KEY RESPONSIBILITIES

• Define and build data quality rules on Databricks spanning out-of-the-box system checks (Tier 1), cross-table/cross-system business logic (Tier 2), and anomaly detection/monitoring (Tier 3)
• Assess and configure the right data quality tooling to sit alongside Databricks-native capability (eg DQX, Great Expectations, Soda) based on technical fit
• Set up Unity Catalog governance tooling for DQ delivery - governed tags, ABAC policies, Lakehouse Monitoring, quality scoring/dashboards
• Convert business-logic requirements from the governance team into working, tested rule sets
• Build and tune anomaly detection thresholds, freshness/completeness monitoring, and alerting/escalation paths
• Document rule logic, thresholds and QC design for handover, audit and ongoing support
• Operate within the underlying AWS environment (S3, IAM) as it relates to DQ implementation
• Work alongside the Senior Data Governance Consultant and client stakeholders to ensure rules align with agreed CDE definitions
• Test, validate and troubleshoot rule configuration ahead of go-live

SKILLS & EXPERIENCE

• Strong hands-on experience building data quality rules on Databricks - Unity Catalog, Lakehouse Monitoring, and/or DQX
• Practical experience with at least one other DQ framework (Great Expectations, Soda or similar), and judgement on which tool fits which requirement
• Strong SQL and/or PySpark for writing DQ rule logic, including cross-table/cross-system checks
• Working AWS experience (S3, IAM, general AWS data architecture) sufficient to operate confidently in a client's AWS-hosted Databricks estate
• Practical understanding of RBAC/ABAC access control models and governed tagging in Unity Catalog
• Comfortable working as a contractor Embedded in a mixed client/consultancy team, with minimal onboarding time

DESIRABLE

• Databricks certifications (eg Data Engineer Associate/Professional)
• Prior experience delivering DQ rule implementation on AWS-hosted Databricks estates
• Monitoring/alerting integration experience (eg CloudWatch, PagerDuty or similar)
• Familiarity with data governance concepts (CDEs, ownership, lineage)