17 of 17 Apache Airflow Jobs in the City of London

bring mission-critical software, analytics, and AI to life and is the company behind Astro, the industry-leading unified DataOps platform powered by Apache Airflow®. Astro accelerates building reliable data products that unlock insights, unleash AI value, and powers data-driven applications. Trusted by more than … Astronomer lets businesses do more with their data. To learn more, visit www.astronomer.io. About this role: At Astronomer, we’re redefining how companies run Apache Airflow at scale. Our R&D organization is home to some of the most innovative minds in cloud infrastructure and open-source software. ...

bring mission-critical software, analytics, and AI to life and is the company behind Astro, the industry-leading unified DataOps platform powered by Apache Airflow®. Astro accelerates building reliable data products that unlock insights, unleash AI value, and powers data-driven applications. Trusted by more than … else, and you have real opinions informed by real failures. Experience working on a SaaS/PaaS product across multiple cloud providers. Familiarity with Apache Airflow or workflow orchestration systems. #LI-Hybrid At Astronomer, we value diversity. We are an equal opportunity employer: we do not discriminate ...

Required Experience Strong hands-on experience with Snowflake, AWS, Python and SQL Proven background in Data Engineering and Data Warehousing Experience with dbt and Airflow Building and supporting enterprise-scale data pipelines CI/CD experience, ideally using GitLab Cloud migration and modern data platform implementation experience Agile delivery ...

environment. Key Responsibilities Design, develop and maintain scalable data pipelines using Python and Databricks. Build, optimise and support enterprise ETL/ELT workflows using Apache Spark and Delta Lake. Design and develop robust data models and Lakehouse architectures. Implement and manage data workflows within the Databricks ecosystem. Ensure high … commercial experience developing data engineering solutions using Python . Extensive hands-on experience with Databricks , including Workflows, Notebooks and Delta Lake. Strong experience with Apache Spark/PySpark . Proven experience building scalable ETL/ELT pipelines. Strong SQL and data modelling skills. Experience designing modern Lakehouse architectures. Experience ...

reviews to maintain high engineering standards. As a Data Engineer you will have the following key skills Cloud: GCP Warehousing: BigQuery/Snowflake Orchestration: Airflow Backend: Python, SQL, Shell Languages: Scala/Java DevOps: Terraform, Docker, Kubernetes Observability: Monte Carlo BI: Looker AI/ML: MLOps London | 12 Months ...

Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure) is a plus Why join High impact and ownership ...

Strong hands‐on background with modern cloud data platforms (AWS/GCP/Azure) Expertise across Python, SQL and data orchestration tools such as Airflow Experience building scalable data pipelines and data platforms in a fast‐paced fintech or technology environment A collaborative leadership style with the ability ...

Strong hands-on background with modern cloud data platforms (AWS/GCP/Azure) Expertise across Python, SQL and data orchestration tools such as Airflow Experience building scalable data pipelines and data platforms in a fast-paced fintech or technology environment A collaborative leadership style with the ability ...

with Snowflake. Knowledge of MongoDB or other NoSQL databases. Experience working within AWS or other cloud environments. Familiarity with workflow orchestration tools such as Airflow, Prefect or Dagster. Experience with market data providers such as Bloomberg, Refinitiv or ICE. Exposure to enterprise data management platforms. Knowledge of Kafka ...

with Snowflake. Knowledge of MongoDB or other NoSQL databases. Experience working within AWS or other cloud environments. Familiarity with workflow orchestration tools such as Airflow, Prefect or Dagster. Experience with market data providers such as Bloomberg, Refinitiv or ICE. Exposure to enterprise data management platforms. Knowledge of Kafka ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

/CD experience. Highly Desirable Familiarity with OneTick, PostgreSQL, ClickHouse, Snowflake, Databricks, kdb+ or similar. Order book reconstruction Tick-to-bar aggregation Airflow, Dagster, Prefect or MLflow Experience supporting quantitative research or trading systems Multi-asset market data (Equities, Futures, FX, Options, ETFs, CFDs) What We're Looking … market data engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience in MLOps is advantageous but not essential. We welcome ...

engineers across the firm. Ideal candidates have deep Python expertise, hands-on experience with modern cluster compute technologies (Kubernetes, Spark, Hadoop/Yarn, Ray, Airflow) and a track record of delivering scalable infrastructure in a multi-user #J-18808-Ljbffr ...

Advanced proficiency in Python, with extensive professional experience in building production applications and services Experience of modern cluster compute - Kubernetes, Spark, Hadoop Yarn, Ray, Airflow are some of the technologies we use Experience leading work in a formal software development environment alongside other engineers in a collaborative manner Strong ...

integration failures, data inconsistencies, and production issues Key Skills Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills Vector databases and embedding pipelines Real-time … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

production‐ready systems that scientists can trust and use daily. You’ll manage and extend a GCP‐first platform, working with tools like BigQuery, Airflow, dbt, and Terraform. You’ll also help build internal applications using Python, FastAPI, and React, giving teams easy ways to trigger workflows and access … Python skills, including building backend services and working with data workflows Experience with GCP, including BigQuery and cloud‐native tools Hands‐on experience with Airflow, dbt, and modern data pipelines Infrastructure as code experience, ideally using Terraform Experience building or contributing to internal tools or applications Strong ...

ADLIB is seeking a Senior Data Platform Engineer to take ownership of their growing platform and help scale it globally. You will work closely with scientists, analysts, and engineers to create scalable systems from complex ...