11 of 11 Apache Airflow Jobs in Central London

PySpark, and dbt Build and optimize batch and streaming data pipelines with strong performance, fault tolerance, and observability Develop and operate workflow orchestration (e.g., Apache Airflow) to schedule, monitor, and manage data movement and transformations Model and transform data for analytics using SQL and dbt to support business … Iceberg, Hudi, or similar) Strong SQL skills and experience with SQL-based transformation tooling (e.g., dbt) Experience designing and operating orchestration pipelines using Airflow or similar tools Experience designing and building streaming pipelines using Kafka, Pub/Sub, or similar messaging systems Preferred qualifications, capabilities, and skills Data modeling ...

operational issues. Understanding of platform security, resilience, scalability and availability. Experience working within Agile and/or DevOps delivery environments . Desirable: Helm, ArgoCD, Airflow, dbt, Bash, Ansible, Packer, Elasticsearch/OpenSearch, Prometheus, Grafana, ELK or Datadog. Experience working within public sector, utilities or other regulated environments is advantageous. ...

principles. Comfortable working within Agile delivery environments. Experience with CI/CD pipelines, ideally using Azure DevOps or similar. Nice to Have Experience with Apache Airflow or dbt. Exposure to BigQuery/GCP. Azure or other cloud platform certifications Knowledge of data governance and data management frameworks Ideal ...

multiple data sources Distributed computing for data intensive work such as Spark and Databricks Cloud platforms including AWS, Azure and GCP Orchestration tools like Airflow Modern DevOps practices including version control, CI/CD, and infrastructure as code using Terraform or Kubernetes Experience building scalable backend applications Understanding ...

compliant solutions that align with GDPR and healthcare data governance standards Stack: Snowflake Snowflake Cortex AWS - S3, Glue, Lambda, Step Functions Python SQL dbt Airflow Terraform Docker GitHub CI/CD Requirements Must have advanced SQL and Python + experience delivering modern cloud-based data platforms Must have enterprise ...

analyse large datasets and communicate insights effectively. Experience with LLMs, generative AI, and prompt engineering. Knowledge of MLOps tools and frameworks (MLflow, Kubeflow, Airflow). Familiarity with SQL, NoSQL, and distributed data technologies. Experience deploying AI models in production environments. Publications, patents, or open-source contributions in ML/ ...

times series data, they use TimescaleDB but other experience with similar platforms is fine You have experience with Orchestration platforms such as Dagster or Airflow You're collaborative with great communication skills What's in it for you: Salary to £85k + bonus X3 weeks per year work anywhere ...

Docker and Kubernetes experience, including deployment and management Previous experience working within air-gapped environments Strong understanding of APIs and microservices Experience with Prefect, Airflow or similar data pipeline/orchestration technologies Experience configuring and integrating workflow tooling such as Jira Experience with database and storage technologies such ...

experience with cloud platforms (AWS, Azure, GCP) Experience in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Practical experience with NLP and/or Search techniques Prompt engineering, and LLMs for enterprise-scale applications. You have team lead ...

with cloud platforms (AWS, Azure, GCP) Well-versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Strong collaboration skills with the ability to work across engineering, research, and product teams across multiple time zones. You have external ...

with cloud platforms (AWS, Azure, GCP) Well versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Practical experience with NLP and/or Search techniques Prompt engineering, and LLMs for enterprise-scale applications. You have team lead ...