Airflow Optimization Specialist (Azure Data Platform)
London, United Kingdom
Hybrid / WFH Options
Hybrid / WFH Options
Thrive IT Systems Ltd
Role: Airflow Optimization Specialist - Azure Data Platform Location: London, UK (Hybrid/Onsite preferred) Type: Contract Start: Immediate Rate: Market competitive (Inside IR35/Outside - depending on experience) We are seeking an Apache Airflow SME to support optimization of our Azure Data Platform . You will play a key role in fine-tuning orchestration, optimizing DAGs … building scalable workflow solutions in a production-grade Airflow environment . Key Responsibilities: Optimize Apache Airflow performance and orchestration pipelines Build modular and scalable DAGs Integrate Airflow with Azure Databricks , Azure Data Factory , and Synapse Implement CI/CD pipelines using Azure DevOps Enable observability with Prometheus , Grafana , Azure Monitor Architect reusable … engineers and provide hands-on leadership Must-Have Skills: 5+ years experience with Apache Airflow (production setup, optimization, plugins) Deep understanding of Airflow scheduler/executors (Celery, Kubernetes) Azure stack: ADF , Databricks , Azure Functions , Synapse , Storage Azure DevOps - YAML pipelines, CI/CD, release management Python (modular/testable coding), Git , Jenkins More ❯
Employment Type: Contract
Rate: GBP 35 Annual
Posted: