Lead PySpark Engineer - Data, SAS, AWS
- Hiring Organisation
- Randstad Digital
- Location
- London, United Kingdom
- Employment Type
- Contract
- Contract Rate
- £350 - £380 per day
Lead Data Engineer - Pyspark/AWS/Python/SAS - Financial Sector As a Lead PySpark Engineer, you will design, develop, and fix complex data processing solutions using PySpark on AWS. You will work hands-on with code, modernising legacy data workflows and supporting large-scale SAS-to-PySpark migrations. … SCDs Spark Performance & Optimisation Strong knowledge of Spark execution concepts, including partitioning, optimisation, and performance tuning. Experience troubleshooting and improving distributed data processing pipelines. Python & Engineering Quality Strong Python coding skills with the ability to refactor, optimise, and stabilise existing codebases. Experience implementing parameterisation, configuration, logging, exception handling, and modular ...