1 to 25 of 80 Apache Airflow Jobs in London

maintain scalable, cloud-based data pipelines and architectures to support advanced analytics and machine learning initiatives. Develop robust ELT workflows using tools like dbt, Airflow, and SQL (PostgreSQL, MySQL) to transform raw data into high-quality, analytics-ready datasets. Collaborate with data scientists, analysts, and software engineers to ensure … however Azure exposure is essential. Data Warehousing: Proven expertise with Snowflake – schema design, performance tuning, data ingestion, and security. Workflow Orchestration: Production experience with Apache Airflow (Prefect, Dagster or similar), including authoring DAGs, scheduling workloads and monitoring pipeline execution. Data Modeling: Strong skills in dbt, including writing modular ...

maintain scalable, cloud-based data pipelines and architectures to support advanced analytics and machine learning initiatives. Develop robust ELT workflows using tools like dbt, Airflow, and SQL (PostgreSQL, MySQL) to transform raw data into high-quality, analytics-ready datasets. Collaborate with data scientists, analysts, and software engineers to ensure … however Azure exposure is essential. Data Warehousing: Proven expertise with Snowflake – schema design, performance tuning, data ingestion, and security. Workflow Orchestration: Production experience with Apache Airflow (Prefect, Dagster or similar), including authoring DAGs, scheduling workloads and monitoring pipeline execution. Data Modeling: Strong skills in dbt, including writing modular ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

solving capabilities. · Excellent communication and documentation skills. · Ability to work independently and lead cross-functional teams. · Adaptability to rapidly evolving technological environments PREFERRED SKILLS · Airflow, dbt, Spark, Kafka, RabbitMQ, Redis. · Git, CI/CD pipelines. · Experience with data warehousing solutions such as Snowflake, Redshift, BigQuery, or Synapse. · Exposure ...

using concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability to communicate technical concepts to both technical ...

Snowflake Production experience with dbt (mandatory) Strong SQL and Python programming skills Experience with Git-based workflows and DevOps practices Familiarity with orchestration tools (Airflow, Prefect) and ETL/ELT patterns Knowledge of cloud platforms (AWS, Azure) and security best practices ...

Snowflake Production experience with dbt (mandatory) Strong SQL and Python programming skills Experience with Git-based workflows and DevOps practices Familiarity with orchestration tools (Airflow, Prefect) and ETL/ELT patterns Knowledge of cloud platforms (AWS, Azure) and security best practices ...

modular, reusable transformations using SQL and Python. Implement CI/CD pipelines and manage deployments via Git. Automate workflows using orchestration tools such as Airflow or dbt Cloud. Configure and optimise Snowflake warehouses for performance and cost efficiency. Required Skills & Experience 7+ years in data engineering roles. 3+ years ...

modular, reusable transformations using SQL and Python. Implement CI/CD pipelines and manage deployments via Git. Automate workflows using orchestration tools such as Airflow or dbt Cloud. Configure and optimise Snowflake warehouses for performance and cost efficiency. Required Skills & Experience 7+ years in data engineering roles. 3+ years ...

intensive experience as a Data Engineer or in a similar role, with a demonstrable track record of leading large-scale projects. Familiarity with Airflow, Dagster or similar data orchestration frameworks Strong understanding of RESTful APIs as well as experience working with both synchronous and asynchronous endpoints Experience with Snowflake ...

intensive experience as a Data Engineer or in a similar role, with a demonstrable track record of leading large-scale projects. Familiarity with Airflow, Dagster or similar data orchestration frameworks Strong understanding of RESTful APIs as well as experience working with both synchronous and asynchronous endpoints Experience with Snowflake ...

technologies (NoSQL, time-series databases) and deep competence in data modeling principles. Workflow Orchestration: Strong working knowledge of industry-standard data orchestration tools (e.g., Apache Airflow or Kubernetes). The Environment We are seeking a highly collaborative professional with strong problem-solving skills, capable of thriving ...

technologies (NoSQL, time-series databases) and deep competence in data modeling principles. Workflow Orchestration: Strong working knowledge of industry-standard data orchestration tools (e.g., Apache Airflow or Kubernetes). The Environment We are seeking a highly collaborative professional with strong problem-solving skills, capable of thriving ...

MLflow), model registry, CI/CD for ML. Serving and orchestration: AWS SageMaker, EKS, batch & Real Time inference. Data pipelines: Spark/SQL, orchestration (Airflow), monitoring for drift/perf. Nice to Have Cost/performance optimisation on AWS; GPU scheduling; quantisation/distillation. Experience working within Financial Services ...

MLflow), model registry, CI/CD for ML. Serving and orchestration: AWS SageMaker, EKS, batch & real-time inference. Data pipelines: Spark/SQL, orchestration (Airflow), monitoring for drift/perf. Nice to Have Cost/performance optimisation on AWS; GPU scheduling; quantisation/distillation. Experience working within Financial Services ...

Familiarity with cloud-based data tools and modern data ecosystems. Understanding of data modeling, warehousing concepts, and best practices Nice-to-Have Experience with Airflow, dbt, or similar orchestration/modeling tools. Experience with BI tools Knowledge of GCP services (Cloud Storage, Dataflow, Pub/Sub, etc.). Social ...

similar) and how they consume data Experience working with cloud environments (AWS, GCP) Familiarity with basic data quality testing and workflow orchestration concepts (e.g., Airflow) Experience working in Agile, cross-functional teams Python experience for automation or scripting would be advantageous ...

similar) and how they consume data Experience working with cloud environments (AWS, GCP) Familiarity with basic data quality testing and workflow orchestration concepts (e.g., Airflow) Experience working in Agile, cross-functional teams Python experience for automation or scripting would be advantageous ...

similar) and how they consume data Experience working with cloud environments (AWS, GCP) Familiarity with basic data quality testing and workflow orchestration concepts (e.g., Airflow) Experience working in Agile, cross-functional teams Python experience for automation or scripting would be advantageous ...

complex or secure environments Strong understanding of data modelling, data integration and data management principles Proficiency in SQL, Python and pipeline tools (e.g., Airflow, Spark) Experience with cloud platforms such as AWS, Azure or GCP Awareness of modern data architectures, analytics and visualisation tooling Strong analytical skills and ability ...

automation processes. Translate complex technical insights into strategic recommendations that inform business decisions and performance. Architect and optimise ELT pipelines and workflows using dbt, Airflow, SQL, and Python. Oversee the design and evolution of large-scale analytics platforms and data warehouses (Snowflake, BigQuery, Redshift). Ensure best practices ...

technologies . You will be utilising and playing with tools across the entire data ecosystem. 🚀Essential tech skills include proficiency in Python, Spark, and Airflow , alongside experience with major data platforms such as Snowflake, Databricks, and BigQuery . 🚀Familiarity with data transformation and infrastructure as code tools like ...

technologies . You will be utilising and playing with tools across the entire data ecosystem. 🚀Essential tech skills include proficiency in Python, Spark, and Airflow , alongside experience with major data platforms such as Snowflake, Databricks, and BigQuery . 🚀Familiarity with data transformation and infrastructure as code tools like ...