5 of 5 Apache Spark Jobs in Wales

team members, building technical expertise and capability Key Responsibilities Develop production-ready data pipelines and processing jobs using batch and streaming frameworks such as Apache Spark and Apache Beam Apply expertise in data storage technologies including relational, columnar, document, NoSQL, data warehouses, and data lakes Implement modern ...

within scalable cloud -based architectures. Robust understanding of data governance, data quality, and metadata management. Desirable skills Experience with Azure Data Factory, Databricks, or Apache Spark, following modern ETL/ELT principles. Experience in using Git, Azure DevOps, or GitHub Actions for version control, CI/ ...

within scalable cloud -based architectures. Robust understanding of data governance, data quality, and metadata management. Desirable skills Experience with Azure Data Factory, Databricks, or Apache Spark, following modern ETL/ELT principles. Experience in using Git, Azure DevOps, or GitHub Actions for version control, CI/ ...

technical delivery of data platforms and pipelines built in AWS-with a focus on AWS Glue, Managed Workflows for Apache Airflow (MWAA), and Python-and collaborate closely with Directors, Senior Architects, and Program Leadership to deliver business outcomes at scale. This is a player-coach role: you will design … environment. Advanced hands-on expertise with: Python (including PySpark & data engineering patterns) AWS Glue (Jobs, Crawlers, Glue Studio, Glue Catalog, PySpark, Job bookmarks) MWAA (Apache Airflow) (DAG design, scheduling, sensors, retries, XComs, task isolation, best practices) Strong across broader AWS services: S3, Lambda, Step Functions, IAM, CloudWatch, KMS, Secrets ...

technical delivery of data platforms and pipelines built in AWS-with a focus on AWS Glue, Managed Workflows for Apache Airflow (MWAA), and Python-and collaborate closely with Directors, Senior Architects, and Program Leadership to deliver business outcomes at scale. This is a player-coach role: you will design … environment. Advanced hands-on expertise with: Python (including PySpark & data engineering patterns) AWS Glue (Jobs, Crawlers, Glue Studio, Glue Catalog, PySpark, Job bookmarks) MWAA (Apache Airflow) (DAG design, scheduling, sensors, retries, XComs, task isolation, best practices) Strong across broader AWS services: S3, Lambda, Step Functions, IAM, CloudWatch, KMS, Secrets ...