Business Consultant - Databricks
- Hiring Organisation
- NTT DATA
- Location
- London, UK
- Employment Type
- Full-time
maintain scalable Databricks pipelines (ETL, streaming, ML workflows). Perform cluster and job performance tuning: optimize cluster sizing, caching, partitioning, and shuffle management. Monitor Spark job metrics, analyze logs, and identify bottlenecks in data throughput or latency. Implement cost-optimization strategies for Databricks jobs and clusters using autoscaling … platform teams to improve query performance on Delta Lake. Develop automated monitoring and alerting mechanisms using Databricks REST APIs, Prometheus, or Azure Monitor. Maintain best practices for data engineering performance, testing, and documentation. Technical Skills Required Strong proficiency with Databricks, Apache Spark (SQL, PySpark, Scala ...