PySpark Developer
- Hiring Organisation
- Randstad Digital
- Location
- London, United Kingdom
- Employment Type
- Contract, Work From Home
- Contract Rate
- £300 - £350 per day
Based) Experience Level: Lead/Senior (5+ years PySpark) Role Overview We are seeking a Lead PySpark Engineer to drive a large-scale data modernisation project, transitioning legacy data workflows into a high-performance AWS cloud environment. This is a hands-on technical role focused on converting legacy … Macros, DI Studio) to PySpark using automated tools (SAS2PY) and manual refactoring. Pipeline Engineering: Design, build, and troubleshoot complex ETL/ELT workflows and data marts on AWS. Performance Tuning: Optimise Spark workloads for execution efficiency, partitioning, and cost-effectiveness. Quality Assurance: Implement clean coding principles, modular design ...