2 of 2 Apache Airflow Jobs in Warwickshire

Solid experience with modern data engineering tools and patterns, with real depth in several of SQL, Spark, Databricks/Delta Lake, orchestration tools (Dagster, Airflow, Temporal), and dbt. Comfort with modern deployment practices: CI/CD, containerisation (Docker), and cloud-based infrastructure. It's a bonus ...

Highly proficient in Python. Highly proficient in version control and Git/GitHub. Experience of workflow management tools, e.g. Nextflow, WDL/Cromwell, Airflow, Prefect, Dagster Understanding of containerisation (e.g. Docker) and deployment (e.g. Kubernetes). Good understanding of cloud environments (ideally Azure), distributed computing and scaling workflows … pipelines Understanding of common data transformation and storage formats, e.g. Apache Parquet. Awareness of data standards such as GA4GH ( ) and FAIR (). Experience with Spark, Databricks, data lakes. Follow best practices like code reviews, clean code and unit tests. Experience working in an agile development team. Why join ...