11 of 11 Permanent Apache Airflow Jobs in Wales

using concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability to communicate technical concepts to both technical ...

Vertex AI or Azure ML Data engineering and MLOps Experience with data warehouses such as Snowflake, BigQuery or Redshift Workflow orchestration using tools like Airflow or Dagster Experience with MLOps tools such as MLflow, Weights and Biases or similar Awareness of data and model drift, and how to monitor ...

Vertex AI or Azure ML Data engineering and MLOps Experience with data warehouses such as Snowflake, BigQuery or Redshift Workflow orchestration using tools like Airflow or Dagster Experience with MLOps tools such as MLflow, Weights and Biases or similar Awareness of data and model drift, and how to monitor ...

using best practices in schema design and data governance. Contribute to ETL/ELT development, testing, and automation using tools such as Python, SQL, Airflow, or Azure Data Factory. Collaborate closely with analysts, scientists, and architects to ensure data solutions meet business needs. Participate in Agile delivery teams, including ...

modelling) ETL/ELT pipeline design, build, and orchestration Docker and containerisation Cloud environments (Azure, AWS, or GCP) Version control (Git) Bonus points for: Airflow, Prefect, dbt, or cloud-native orchestration CI/CD and deployment automation Data warehousing fundamentals (star schema, staging patterns) Plus: Ability to explain complex ...

Organization. Technologies used on the team include: Languages: Go, Python, Scala Frameworks: Spark, Thrift, Kafka Data stores: Postgres, BigQuery, Redis, Druid Tools: Kubernetes, Airflow, Docker Job responsibilities: You will work with the existing engineering team and help contribute to the direction of the projection and team Collaborate with cross ...

start-ups or high-growth scale-ups • Strong technical grounding across Node.js, Typescript, React, MySQL and cloud-native architectures (AWS, ECS/K8s, Kafka, Airflow) • Comfortable driving Agile and DevOps culture, and setting high standards for clean, testable, secure code in fast-moving environments If you want ...

resilience and data residency Confidence working with senior stakeholders to make pragmatic, commercial platform decisions Tech environment includes: AWS, Azure, Kubernetes (EKS), ECS, Terraform, Airflow, EFK/Prometheus/Grafana, MySQL/PostgreSQL/Redis/Elasticsearch/Kafka, GitHub Actions, plus services written in Python, Node.js and Go. ...

resilience and data residency Confidence working with senior stakeholders to make pragmatic, commercial platform decisions Tech environment includes: AWS, Azure, Kubernetes (EKS), ECS, Terraform, Airflow, EFK/Prometheus/Grafana, MySQL/PostgreSQL/Redis/Elasticsearch/Kafka, GitHub Actions, plus services written in Python, Node.js and Go. ...

fast-paced, challenging environment. Key Responsibilities: Design, implement, and debug ETL pipelines to process and manage complex datasets. Leverage big data tools, including Apache Kafka, Spark, and Airflow, to deliver scalable solutions. Collaborate with stakeholders to ensure data quality and alignment with business goals. Utilize programming expertise … data engineering projects Good hands-on experience of designing, implementing, debugging ETL pipeline Expertise in Python, PySpark and SQL languages Expertise with Spark and Airflow Experience of designing data pipelines using cloud native services on AWS Extensive knowledge of AWS services like API Gateway, Lambda, Redshift, Glue, Cloudwatch, etc. ...

fast-paced, challenging environment. Key Responsibilities: Design, implement, and debug ETL pipelines to process and manage complex datasets. Leverage big data tools, including Apache Kafka, Spark, and Airflow, to deliver scalable solutions. Collaborate with stakeholders to ensure data quality and alignment with business goals. Utilize programming expertise … data engineering projects Good hands-on experience of designing, implementing, debugging ETL pipeline Expertise in Python, PySpark and SQL languages Expertise with Spark and Airflow Experience of designing data pipelines using cloud native services on AWS Extensive knowledge of AWS services like API Gateway, Lambda, Redshift, Glue, Cloudwatch, etc. ...