3 of 3 Apache Airflow Jobs in the City of Westminster

PySpark, and dbt Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observability Develop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformations Model and transform data for analytics using SQL and dbt to support business … Iceberg, Hudi, or similar) Strong SQL skills and experience with SQL-based transformation tooling (e.g., dbt) Experience designing and operating orchestration pipelines using Airflow or similar tools Experience designing and building streaming pipelines using Kafka, Pub/Sub, or similar messaging systems Preferred qualifications, capabilities, and skills Data modeling ...

product partners Preferred Qualifications, Capabilities, and Skills Experience building agentic tools and AI agents that support engineering workflows in production environments Experience with Airflow ABOUT US Our client is a global leader in financial services, providing strategic advice and products to the world's most prominent corporations, governments, wealthy ...

design, API development, and data modelling for production machine learning systems. Build and operate production solutions using technologies such as Python, TensorFlow, PyTorch, Redis, Airflow, Docker, Kubernetes, and cloud-native infrastructure. Design and deploy scalable batch and real-time ML pipelines supporting customer-facing applications. Develop and operationalise … designing, building, or scaling RAG systems. Experience in search, ranking, recommendations, personalisation, content intelligence, moderation, NLP, computer vision, or related AI domains. Experience with Airflow or comparable workflow orchestration platforms. Experience with cloud-native environments and container technologies, including Docker and Kubernetes. Experience developing model evaluation frameworks ...