3 of 3 Permanent Amazon EMR Jobs in the North East

data processing workflows. Root cause analysis Perform data analysis to identify and resolve root causes of pipeline failures and data quality issues - including reviewing EMR output logs and CloudWatch metrics. Apply understanding of dimensional data models and slowly changing dimensions (SCD) to design and maintain well-structured, analytically trusted … development and data processing SQL - used for querying, transformation, and validation across data stores PySpark - for distributed data processing using Apache Spark on AWS EMR Familiarity with basic data structures for constructing robust, scalable solutions Data processing & orchestration Apache Spark - understanding of distributed data processing architecture and execution Apache ...

Experience working with AWS and its data and AI/ML services such as SageMaker - Experience of big data and lakehouse platforms on AWS (EMR, Databricks or Snowflake) - Experience building data platforms such as Data Lakes, Lakehouses or Data Meshes that underpin analytics and AI workloads - Understanding of Data ...

Experience: Understanding of data processing using Apache Spark and Apache Airflow for task orchestration Use of Python, SQL, and familiarity with PySpark Understanding of EMR and reviewing output logs Use of Jupyter notebooks and/or Amazon Athena to query and validate data Data analysis to identify … changing dimensions/historic data capture Use of AWS console and services such as, but not limited to; CloudWatch, IAM, S3, Glue, ECR, EC2, EMR, Dynamo DB and LakeFormation Familiarity with Amazon Textract and Comprehend Understanding of both server-side and client-side encryption Use of GitLab ...