3 of 3 Permanent Amazon EMR Jobs in Tyne and Wear

data processing workflows. Root cause analysis Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues including reviewing EMR output logs and CloudWatch metrics. Data modelling Apply understanding of dimensional data models and slowly changing dimensions (SCD) to design and maintain well-structured … development and data processing SQL used for querying, transformation, and validation across data stores PySpark for distributed data processing using Apache Spark on AWS EMR Familiarity with basic data structures for constructing robust, scalable solutions Data processing & orchestration Apache Spark understanding of distributed data processing architecture and execution Apache ...

nice to have: Data Engineering & Programming: • Python • SQL • Spark/PySpark • Scala • Data modelling and warehousing concepts Cloud & Platforms: • AWS (S3, Redshift, Glue, EMR, Lambda) • Other cloud platforms (Azure/GCP considered) • Data lake and data warehouse architectures General: • Strong problem-solving skills • Experience working in agile environments • Good ...

large-scale cloud data engineering programme, operating across a modern AWS-native technology stack including Apache Airflow, Amazon Athena, AWS Glue, S3, EMR, and DynamoDB. You will own testing across automated pipelines, data workflows, and cloud infrastructure - identifying risks, championing test frameworks, and coaching colleagues in quality … test frameworks Enhance existing frameworks to improve testing confidence and coverage Pipeline & Data Testing Validate data pipelines using Apache Airflow, AWS Glue, Athena, and EMR Ensure data integrity, transformation accuracy, and performance under load Analyse data in multiple formats to validate new functionality Perform production data analysis to identify ...