Data Engineer (SC Cleared)
- Hiring Organisation
- scrumconnect ltd
- Location
- City, Newcastle Upon Tyne, United Kingdom
- Employment Type
- Any
- Salary
- GBP Annual
practices and the ability to translate customer expectations into applied technical functionality are key to success in this role. Key responsibilities Data pipeline development Build and maintain scalable data pipelines using Apache Spark and PySpark, processing and transforming large datasets across distributed cloud infrastructure. Workflow … monitoring, and execution of data processing workflows. Root cause analysis Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues including reviewing EMR output logs and CloudWatch metrics. Data modelling Apply understanding of dimensional data ...