Data Engineer
- Hiring Organisation
- DWP Digital
- Location
- Newcastle Upon Tyne, Tyne and Wear, North East, United Kingdom
- Employment Type
- Permanent, Work From Home
- Salary
- £50,000
/CD delivery practices. Ability to optimise data processing, storage, and queries to improve performance and data quality. Experience designing and automating ETL pipelines using PySpark and Spark SQL.A strong full stack development background with technologies that include .NET 4,8, C#, JavaScript, C++ and Java. You and your role … will design, build, maintain, and optimise scalable data products used by our analysts and data scientists. You will develop and orchestrate PySpark-based ETL pipelines, using technologies such as Amazon EMR, AWS Lambda, and EC2 to process large volumes of structured and unstructured data efficiently. Your role involves collaborating with ...