13 of 13 Apache Airflow Jobs in the City of London

automated testing and data quality practice. A collaborative mindset and a genuine interest in building things well. Desirable Experience with orchestration tools such as Airflow, Dagster, Prefect or Azure Data Factory. Experience working within AWS, Azure, GCP or another enterprise cloud ecosystem. Experience with complex customer, transactional, digital ...

Shape the engineering approach across Google Cloud Platform (GCP) , working with technologies including BigQuery, Cloud Storage, Pub/Sub, Datastream and Cloud Composer/Airflow, alongside Terraform and infrastructure-as-code. Set and champion high engineering standards across data modelling, dbt (data build tool) transformations, testing, monitoring, documentation, deployment … BigQuery, and a good working knowledge of the wider Google Cloud Platform (GCP) ecosystem. Experience with modern data engineering tools and practices such as Airflow/Cloud Composer, dbt, Git/GitHub, CI/CD and Terraform or other infrastructure-as-code tooling . A solid understanding of data ...

live ML models. BAU Operator: High autonomy to own daily operations and deliver under tight deadlines. Cloud & BI Stack: Practical exposure to GCP (BigQuery), Airflow, or Looker . AI Tooling: Familiarity with Claude Code, OpenAI, Cursor , or agentic workflows to accelerate delivery. Compliance & Domain: Prior experience with FCA reporting ...

economy. Familiarity with Supabase specifically, or with similar BaaS platforms. BigQuery experience. Familiarity with n8n. Equivalent experience with code‐first orchestrators such as Temporal, Airflow or Prefect is welcome. Design sensibility. You care how things look and feel, not just how they function. Experience replacing legacy or external vendor ...

CDO. What you’ll need for this role Hands-on experience designing and operating large-scale data pipelines on GCP (BigQuery, Cloud Composer/Airflow, GCS), with proven knowledge of Apache Kafka and real-time streaming architectures Demonstrated experience leading or technically mentoring a team of data engineers ...

/Java and/or Python. Experience managing batching and/or streaming data pipelines and infrastructure using technologies such as GCP, Dataflow/Apache Beam, Dataproc, Spark/Flink, dbt; Working knowledge of SQL and shell scripting; Experience with Docker/k8s and "infrastructure as a code" tools … partners across the organization via Technical Design Documents. This is a great opportunity to further your experience across technologies such as Cloud Composer, Airflow, Dataflow, Spark, Terraform & Google Cloud Platform. Benefits Hybrid working Long contract Award winning work environment #J-18808-Ljbffr ...

data infrastructure, while contributing to strong engineering practices and standards. The environment is heavily focused on Google Cloud Platform (GCP), with BigQuery, Dataform and Airflow/Cloud Composer forming key parts of the technology stack. WHAT YOU'LL BE DOING * Designing and implementing complex data flows across operational, analytics … clearly defined. ESSENTIAL EXPERIENCE * Strong hands-on experience with Google Cloud Platform (GCP). * Strong experience with BigQuery. * Experience using Dataform. * Experience with Airflow/Cloud Composer. * Strong data engineering fundamentals, including building scalable and robust production data solutions. * Experience working with data quality, governance and appropriate data controls. ...

integration failures, data inconsistencies, and production issues Key Skills Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills Vector databases and embedding pipelines Real-time … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

messy data environments. Naturally curious who will dig a level deeper into the data and question assumptions. Desirable Skills (Nice to Have) Snowflake, DBT, Airflow: Practical experience working with Snowflake, DBT and Airflow Trayport is committed to creating and sustaining a collegial work environment in which all individuals ...

Implement data-quality checks, testing and documentation Work with distributed data technologies such as Presto, Spark or Hive Use workflow orchestration tools such as Airflow Participate in code reviews and Git-based development workflows Collaborate with Data Engineers, Analysts and Engineering teams The core work is production data engineering … data pipelines Advanced SQL , including performance optimisation on large tables Strong Python for data engineering Experience with Presto, Spark, Hive or similar Experience with Airflow or another workflow scheduler Strong ETL/data pipeline development experience Git and code-review experience Strong understanding of data quality and accuracy Experience ...

driven user experience. Hybrid role with three on-site days in City of London. Responsibilities include building scalable data pipelines (BigQuery, Dataflow/Apache Beam, Airflow), ensuring data quality and observability (Looker, Monte Carlo), and collaborating with product engineering and data science teams to plan data tracking ...

SENIOR SCALA ENGINEER £590 per day via Umbrella 6-12 month contract Stratford, London – hybrid remote working SC clearance required ABOUT THE CLIENT Our client is a major UK public sector organisation delivering digital services ...

Trayport is seeking a capable BI professional to join our expanding BI team in London. You will turn messy data into clear insights for stakeholders, bridging data infrastructure and business units. The role emphasizes communication ...