Azure Databricks Engineer
- Location
- Greater London, England, United Kingdom
Lake developmentCluster management, compute policies, autoscaling, and workload monitoringData EngineeringStrong Python, PySpark, and SQL development skillsDesign and build ingestion and transformation pipelinesFull and incremental data loadingCDC (Change Data Capture) implementationData quality controls, schema evolution, reconciliation, and error handlingIntegration DevelopmentREST API integration experienceIntegration with databases, SaaS applications, cloud storage … governance and platform standards implementationDiscovery Zone & Migration SupportExperience enabling application deployment within DatabricksBI tooling integration and schedulingSupport for POSIT/RStudio migration activitiesCollaborationWorking with Data Engineers, Data Scientists, and Platform TeamsTechnical documentation and knowledge transferAgile delivery experienceDesirable SkillsPOSIT/RStudio migration experienceAI/ML and LLM integrationRAG ...