3 of 3 Permanent Jupyter Jobs in Newcastle upon Tyne

solutions Data processing & orchestration Apache Spark - understanding of distributed data processing architecture and execution Apache Airflow - configuring DAGs and managing task orchestration at scale Jupyter Notebooks - for exploratory data analysis and pipeline prototyping Understanding of dimensional data models and slowly changing dimensions (SCD Types 1, 2, 3) Data analysis skills ...

familiarity with PySpark Experience with Apache Spark and data processing frameworks Hands-on experience with Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/or Amazon Athena for data querying and validation Strong understanding of EMR and log analysis Experience with GitLab for CI/CD pipelines, release ...

Glue, AWS EMR, AWS EC2, AWS ECR, AWS DynamoDB, AWS CloudWatch, AWS IAM, Python, SQL, Java, Bash, GitLab CI/CD, Jupyter Notebooks, Apache Spark/PySpark, Terraform, Docker Key Skills Strong proficiency in Python, SQL, Java, and Scripting languages such as Bash Hands-on experience testing Big Data platforms … Data Lake solutions Strong experience with AWS EMR, Apache Airflow, and Jupyter Notebooks Experience performing PySpark-based data interrogation, validation, and testing Strong knowledge of AWS services including S3, Glue, Athena, EMR, CloudWatch, IAM, DynamoDB, EC2, and ECR Experience designing, implementing, and managing enterprise-scale automated test frameworks Strong understanding ...