Lead Azure Databricks Platform Engineer / Architect
- Hiring Organisation
- TXP
- Location
- London, United Kingdom
- Employment Type
- Contract, Work From Home
- Contract Rate
- £500 - £501 per day
Engineering and Integration Design and build reliable ingestion and transformation pipelines using Python, PySpark, SQL and Delta Lake. Implement full and incremental ingestion, CDC where appropriate, schema evolution, reconciliation, error handling and data quality controls. Design reusable integration patterns for REST APIs, SaaS platforms, databases, files, object storage … identity, networking, security, secrets, monitoring and private connectivity Databricks SQL, Delta Lake and performance optimisation Python, PySpark and SQL Jobs/workflows, incremental processing, CDC and data quality REST/API and external data integration patterns FinOps, cost attribution, tagging, budgets, monitoring and operational support Experience and Candidate ...