Lead Compute Platform Engineer
- Location
- Greater London, England, United Kingdom
that simplifies the execution of Spark jobs,facilitatingthe analysis of terabytes of data with ease and cost-effectiveness. The runtime environment is based on Hadoop clusters on EC2 spot instances, dynamically scaling up asrequired, ensuring high resourceutilizationand stability. Our teamis responsible forthe development and deployment of the company … framework and services Identifyingand implementing computational improvements Making improvements and/or fixing bugs bysubmittingpatches toopen-sourceprojects in our stack, such as Spark,Iceberg,Hadoop, Airflow, and many others Acting as anexpert on Spark and Scala Mentoring engineers (including through pairing, design discussions, and knowledge-sharing sessions) and helping ...