7 of 7 Apache Airflow Jobs in the South West

focus of the role will be supporting our transition from Matillion to a modern Data Engineering framework built on Snowflake, dbt and Apache Airflow. This is an ideal opportunity for a hands-on Data Engineer who enjoys solving complex data challenges, building scalable solutions and working within a modern … pipelines and data transformation processes. Developing SQL-based solutions within our Snowflake data platform. Supporting the migration of existing Matillion workflows to dbt and Apache Airflow. Building and optimising ELT processes to improve efficiency, scalability and data quality. Working with stakeholders to gather requirements and translate business needs into ...

Python skills * Hands-on experience designing and developing ETL pipelines and data workflows * Experience designing and maintaining data pipelines using technologies such as Apache Airflow or Spark * Experience working with cloud platforms including AWS, Azure or GCP * Strong understanding of data modelling, integration and governance principles * Experience working ...

applying DevSecOps practices within cloud or controlled environments Exposure to large language models or generative AI systems Experience with data engineering tools (e.g. Spark, Airflow, Databricks) Experience applying AI and automation to operational or productivity-focused use cases Experience working in secure, restricted, or air-gapped environments Experience supporting ...

well-governed data products. Job responsibilities Design and implement batch and streaming data pipelines using Databricks (Spark), Delta Lake , and orchestrators (e.g., Workflows , Airflow, ADF). Develop and optimize Spark jobs (PySpark/Scala) and SQL transformations for performance, reliability, and cost efficiency. Build and maintain curated data models … automation capabilities, to improve the value realized by automation Required qualifications, skills and capabilities Experience with building data pipelines on Databricks and/or Apache Spark in production. Strong coding skills in Python (PySpark) and SQL (Scala a plus). Hands-on experience with Delta Lake (MERGE/UPSERT ...

given real ownership, exposure to experienced director level mentors & Senior Data Engineers, alongside the opportunity to grow towards senior-level responsibilities. Tech Stack Spark | Airflow | Kubernetes | Azure | CubeJS What You'll Be Doing Data Pipeline Development - designing & developing efficient, scalable & robust Data Pipelines Support and main the data platform ...

deployments Troubleshoot integration failures, data inconsistencies, and production issues Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills Vector databases and embedding pipelines Real-time … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

integration failures, data inconsistencies, and production issues Key Skills Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills Vector databases and embedding pipelines Real-time … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...