4 of 4 Apache Airflow Jobs in the Thames Valley

code Requirements: Must have strong, commercial AWS (contract) Data Engineering experience Must bring solid knowledge of data warehousing and modelling Experience with Airflow and infrastructure as a code (Terraform, CloudFormation) would be beneficial Must have experience with Redshift Nice to have: Experience with dbt/lake formation ...

PostgreSQL. Some exposure to networked services and APIs, such as gRPC or REST. Exposure to schedulers, job queues or workflow engines - Celery, Airflow, Temporal, Slurm or similar. Comfort working on Linux, and debugging a service that is already running. Good communication skills both within software and across disciplines. ...

machines, timeouts, leases, retries, and the failure modes that only show up under load. Experience with a scheduler, workflow engine or job queue - Temporal, Airflow, Celery, Slurm, or one you built - or with event‐driven systems that react to changes in state. A habit of settling design questions ...

reach for determinism when chasing a timing or interleaving bug, rather than for more logging. Depth in the scheduling and workflow landscape - Temporal, Hatchet, Airflow, Slurm, Postgres-backed queues - and the judgement to borrow from it rather than reinvent. A track record of pinning down failures that others could ...