3 of 3 Amazon Athena Jobs in Newcastle upon Tyne

ensuring data pipelines, processing workflows and outputs are accurate, reliable and fit for purpose. You will use tools such as Apache Airflow, Jupyter Notebooks, Amazon Athena and AWS EMR to execute, run, query and monitor data processes. You will also use SQL, Python and PySpark to interrogate … unstructured data Interrogate and validate data using SQL, Python and PySpark Execute, run, query and monitor data processes using Apache Airflow, Jupyter Notebooks, Amazon Athena and AWS EMR Validate data ingestion, transformation, processing and output across AWS-based data platforms Identify data quality issues, reconciliation errors, pipeline failures ...

quality assurance and testing activities within a large-scale cloud data engineering programme operating across a modern AWS-native technology ecosystem, including Apache Airflow, Amazon Athena, AWS Glue, S3, EMR, DynamoDB, and related cloud services. This role combines hands-on technical leadership with test strategy ownership . … Support infrastructure testing and validation within Terraform-managed cloud environments Contribute to continuous improvement initiatives across testing and delivery processes Technology Stack Apache Airflow, Amazon Athena, AWS S3, AWS Glue, AWS EMR, AWS EC2, AWS ECR, AWS DynamoDB, AWS CloudWatch, AWS IAM, Python, SQL, Java, Bash, GitLab ...

Experience with Apache Spark and data processing frameworks Hands-on experience with Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/or Amazon Athena for data querying and validation Strong understanding of EMR and log analysis Experience with GitLab for CI/CD pipelines, release tagging … version control Knowledge of Docker and containerisation Understanding of encryption techniques (Server Side and client-side) Familiarity with Amazon Textract and Comprehend Strong understanding of data modelling concepts, including dimensional models and slowly changing dimensions Additional Skills: Ability to translate customer requirements into functional technical solutions Familiarity with engineering ...