26 to 50 of 103 Apache Airflow Jobs in London

planning, and UAT support. Experience with cloud‐based data platforms, particularly Snowflake. Understanding of ETL/ELT frameworks and tools such as dbt and Apache Airflow. Knowledge of data governance, metadata management, and data lineage concepts. Experience supporting Business Intelligence and reporting solutions. Familiarity with modern analytics engineering practices. ...

approaches, fit to the use case Build and maintain transformation layers in dbt (or equivalent), with appropriate testing, documentation and lineage Operate orchestration tooling (Airflow, Dagster, Prefect or equivalent) and the workflows around it Manage cloud data warehouses and lakehouses (Snowflake, BigQuery, Redshift, Synapse, Databricks) Implement data quality, testing … Python skills (or Scala/Java where relevant) for pipeline development and tooling Hands-on experience with dbt or equivalent transformation tooling Orchestration experience (Airflow, Dagster, Prefect or equivalent) Production experience with at least one major cloud warehouse/lakehouse (Snowflake, BigQuery, Redshift, Synapse, Databricks) Cloud platform experience ...

Proven experience in a Lead/Senior Data Engineering role Strong handson experience with Snowflake Expertise in ETL/ELT and orchestration tools (e.g. Airflow, dbt, Matillion) Strong SQL and Python skills Experience building scalable data platforms in cloud environments Previous experience leading or mentoring engineers Strong communication ...

Snowflake Toolkit: Hands-on with Snowpipe, Streams, Tasks, Dynamic Tables, and Stored Procedures. Nice to Have: Exposure to DBT, Snowpark, Snowflake AI/ML, Airflow, and Snowflake administration (RBAC, warehouse sizing). Mindset: Strong problem-solving abilities with an ownership mindset for end-to-end delivery. This ...

Strong cloud data platform expertise, including Google Cloud Platform (GCP). Hands-on experience with BigQuery, Dataflow, Datastream, Data Fusion, Cloud Functions, Dataproc, and Airflow/Composer. Advanced SQL and Python skills. Expertise in data lakes, data warehouses, and modern data architectures. Strong understanding of data governance, lineage ...

orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic and transparency of underlying code Owned a project across a whole ...

technologies and practices such as Docker, Terraform, CI/CD pipelines and infrastructure‐as‐code. Familiarity with MLOps tools and practices such as MLflow, Airflow, model monitoring, API monitoring, data validation, data drift detection, autoscaling and access management. Strong Python experience, with helpful knowledge of Spark or PySpark ...

such as Streamlit, FastAPI, or Flask, enabling business stakeholders to interact directly with model outputs. Experience with orchestration or transformation tooling such as dbt, Airflow, or Prefect, demonstrating an ability to build reproducible and maintainable pipelines. A tech-agnostic mindset, open to selecting the right tool for each problem ...

such as Streamlit, FastAPI, or Flask, enabling business stakeholders to interact directly with model outputs. Experience with orchestration or transformation tooling such as dbt, Airflow, or Prefect, demonstrating an ability to build reproducible and maintainable pipelines. A tech-agnostic mindset, open to selecting the right tool for each problem ...

such as Streamlit, FastAPI, or Flask, enabling business stakeholders to interact directly with model outputs. Experience with orchestration or transformation tooling such as dbt, Airflow, or Prefect, demonstrating an ability to build reproducible and maintainable pipelines. A tech-agnostic mindset, open to selecting the right tool for each problem ...

such as Streamlit, FastAPI, or Flask, enabling business stakeholders to interact directly with model outputs. Experience with orchestration or transformation tooling such as dbt, Airflow, or Prefect, demonstrating an ability to build reproducible and maintainable pipelines. A tech-agnostic mindset, open to selecting the right tool for each problem ...

preferred) or similar platforms: Snowflake, BigQuery, Redshift, Synapse, or equivalent Strong proficiency with: SQL (advanced level) Spark/PySpark Data pipeline orchestration tools (e.g., Airflow, ADF, etc.) Data Modelling Extensive experience designing: Fact and dimension tables Star and snowflake schemas Aggregation layers and semantic models Data Migration & Mapping Demonstrated ...

operating production data systems Strong SQL and Python skills with expertise in large-scale data processing Hands-on experience with modern data stack tools — Airflow, dbt, ClickHouse, BigQuery, Snowflake, or similar Experience with event-driven, real-time or near real-time architectures A first-class approach to data correctness ...

with Snowflake. Knowledge of MongoDB or other NoSQL databases. Experience working within AWS or other cloud environments. Familiarity with workflow orchestration tools such as Airflow, Prefect or Dagster. Experience with market data providers such as Bloomberg, Refinitiv or ICE. Exposure to enterprise data management platforms. Knowledge of Kafka ...

implementing batch and stream data processing systems. Experience with Datadog, SQL, Terraform, Docker, AWS (ECS, Kinesis, SQS, CloudWatch), GCP (BigQuery). Knowledge of Airflow, Kafka, Flink, dbt, Kubernetes is highly beneficial. Good understanding of AI platforms, experience building infrastructure for LLM orchestration (e.g., vector databases, RAG pipelines, GPU scaling ...

with Snowflake. Knowledge of MongoDB or other NoSQL databases. Experience working within AWS or other cloud environments. Familiarity with workflow orchestration tools such as Airflow, Prefect or Dagster. Experience with market data providers such as Bloomberg, Refinitiv or ICE. Exposure to enterprise data management platforms. Knowledge of Kafka ...

schemas to support business requirements Develop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving ...

schemas to support business requirements Develop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving ...

schemas to support business requirements Develop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving ...

support business requirements \n Develop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. \n Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. \n Working collaboratively with … PostgreSQL) \n You have framework experience within either Flask, Tornado or Django, Docker \n Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo \n Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. \n Data acquisition and development of data sets ...

Strong hands‐on background with modern cloud data platforms (AWS/GCP/Azure) Expertise across Python, SQL and data orchestration tools such as Airflow Experience building scalable data pipelines and data platforms in a fast‐paced fintech or technology environment A collaborative leadership style with the ability ...

Strong hands-on background with modern cloud data platforms (AWS/GCP/Azure) Expertise across Python, SQL and data orchestration tools such as Airflow Experience building scalable data pipelines and data platforms in a fast-paced fintech or technology environment A collaborative leadership style with the ability ...

notebooks for analysis dbt and Databricks data platform with Delta Lake Lightdash as our BI tool and semantic layer Streamlit for interactive data applications Airflow for orchestration Kubernetes for application deployment CI/CD with GitHub and CircleCI Infrastructure on AWS, deployed with Terraform/Spacelift ...

notebooks for analysis dbt and Databricks data platform with Delta Lake Lightdash as our BI tool and semantic layer Streamlit for interactive data applications Airflow for orchestration Kubernetes for application deployment CI/CD with GitHub and CircleCI Infrastructure on AWS, deployed with Terraform/Spacelift ...

platforms and pipelines (batch and real-time ingestion). Hands-on familiarity with: Snowflake (or equivalent cloud data warehouse), Looker (or other BI tools), Airflow, Pub/Sub, or similar orchestration and ingestion frameworks. Experience with data infrastructure monitoring and optimization. Strong communication skills. Ability to translate business goals ...