1 to 25 of 54 Apache Airflow Jobs in London

maintain scalable, cloud-based data pipelines and architectures to support advanced analytics and machine learning initiatives. Develop robust ELT workflows using tools like dbt, Airflow, and SQL (PostgreSQL, MySQL) to transform raw data into high-quality, analytics-ready datasets. Collaborate with data scientists, analysts, and software engineers to ensure … however Azure exposure is essential. Data Warehousing: Proven expertise with Snowflake – schema design, performance tuning, data ingestion, and security. Workflow Orchestration: Production experience with Apache Airflow (Prefect, Dagster or similar), including authoring DAGs, scheduling workloads and monitoring pipeline execution. Data Modeling: Strong skills in dbt, including writing modular ...

similarly regulated industry. Strong coding skills in Python or Java (Spring Boot); React experience is a plus. Proficiency with modern data tools: Airflow, Spark, Kafka, dbt, Snowflake or similar. Experience with cloud platforms (AWS, Azure, GCP), containerization (Docker, Kubernetes), and CI/CD. Data Quality: Proven ability to validate ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

solving capabilities. · Excellent communication and documentation skills. · Ability to work independently and lead cross-functional teams. · Adaptability to rapidly evolving technological environments PREFERRED SKILLS · Airflow, dbt, Spark, Kafka, RabbitMQ, Redis. · Git, CI/CD pipelines. · Experience with data warehousing solutions such as Snowflake, Redshift, BigQuery, or Synapse. · Exposure ...

Data Engineering (not building pipelines but designing and building the framework) DevOps MLOps/LLMOps Often work with the following technologies : Azure, AWS, GCP Airflow, dbt, Databricks, Snowflake, etc. GitHub, Azure DevOps and related developer tooling and CI/CD platforms, Terraform or other Infra-as-Code MLflow, AzureML ...

using concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability to communicate technical concepts to both technical ...

Snowflake Production experience with dbt (mandatory) Strong SQL and Python programming skills Experience with Git-based workflows and DevOps practices Familiarity with orchestration tools (Airflow, Prefect) and ETL/ELT patterns Knowledge of cloud platforms (AWS, Azure) and security best practices ...

modular, reusable transformations using SQL and Python. Implement CI/CD pipelines and manage deployments via Git. Automate workflows using orchestration tools such as Airflow or dbt Cloud. Configure and optimise Snowflake warehouses for performance and cost efficiency. Required Skills & Experience 7+ years in data engineering roles. 3+ years ...

intensive experience as a Data Engineer or in a similar role, with a demonstrable track record of leading large-scale projects. Familiarity with Airflow, Dagster or similar data orchestration frameworks Strong understanding of RESTful APIs as well as experience working with both synchronous and asynchronous endpoints Experience with Snowflake ...

technologies (NoSQL, time-series databases) and deep competence in data modeling principles. Workflow Orchestration: Strong working knowledge of industry-standard data orchestration tools (e.g., Apache Airflow or Kubernetes). The Environment We are seeking a highly collaborative professional with strong problem-solving skills, capable of thriving ...

MLflow), model registry, CI/CD for ML. Serving and orchestration: AWS SageMaker, EKS, batch & real-time inference. Data pipelines: Spark/SQL, orchestration (Airflow), monitoring for drift/perf. Nice to Have Cost/performance optimisation on AWS; GPU scheduling; quantisation/distillation. Experience working within Financial Services ...

days/week from Central London office Strong problem-solving skills, attention to detail, and ability to work independently Experience with workflow orchestration (Prefect, Airflow, Dagster), cloud data warehouses (BigQuery, Snowflake), or backend Python frameworks (Django, FastAPI) is a plus Benefits 25 days annual leave plus bank holidays Meaningful ...

similar) and how they consume data Experience working with cloud environments (AWS, GCP) Familiarity with basic data quality testing and workflow orchestration concepts (e.g., Airflow) Experience working in Agile, cross-functional teams Python experience for automation or scripting would be advantageous ...

similar) and how they consume data Experience working with cloud environments (AWS, GCP) Familiarity with basic data quality testing and workflow orchestration concepts (e.g., Airflow) Experience working in Agile, cross-functional teams Python experience for automation or scripting would be advantageous ...

downstream users consume data Experience working in cloud environments (AWS or GCP) Familiarity with data quality testing, deployment workflows, and orchestration concepts (e.g. Airflow) Experience working in Agile, cross-functional product or platform teams Python experience for automation or data tooling is a plus ...

with hands on Snowflake experience • Proven dbt experience at scale • Strong dimensional modelling using Kimball principles • Python for data engineering tasks • Experience with Airflow or similar orchestration tools • Git based workflows • Familiarity with BI tools such as Tableau, Looker, or Power BI • Experience working in cloud environments including ...

with hands on Snowflake experience* Proven dbt experience at scale* Strong dimensional modelling using Kimball principles* Python for data engineering tasks* Experience with Airflow or similar orchestration tools* Git based workflows* Familiarity with BI tools such as Tableau, Looker, or Power BI* Experience working in cloud environments including ...

technologies . You will be utilising and playing with tools across the entire data ecosystem. 🚀Essential tech skills include proficiency in Python, Spark, and Airflow , alongside experience with major data platforms such as Snowflake, Databricks, and BigQuery . 🚀Familiarity with data transformation and infrastructure as code tools like ...

have indepth technical knowledge of Python for data workflows, SQL and Data Modelling, DBT, Cloud platforms (AWS, GCP or Azure), Workflow Orchestration tools (e.g. Airflow, Dagster, Prefect, DBT Cloud), CI/CD (GitHub Actions, GitLab CI, CircleCI) You have experience of leading technical deployments, running customer projects, diagnosing issues ...

pipelines in a production environment. Exposure to cloud-based data platforms (AWS, GCP, Azure, or similar). Familiarity with data orchestration tools (e.g. Airflow or equivalent). Solid understanding of data warehousing concepts, schema design, and data modelling. Oscar Associates (UK) Limited is acting as an Employment Agency ...

investment professionals, and who thrives in fast-paced financial environments... Key Responsibilities Help users adopt and make effective use of AWS, Kubernetes, Terraform, Airflow, Jenkins, Coder, and other cloud/platform technologies Streamline operational workflows by automating routine tasks, enhancing documentation, and building out knowledge bases Work with engineering ...

another language e.g. Python ELT/ETL CI/CD Data Modelling Data Architecture Data Frameworks dbt Data processing and orchestration using Airflow or similar Google BigQuery or Redshift Cloud based services: GCP (preferred) or AWS Agentic development frameworks Automations About You 6 years+ experience as a Data Engineer ...

peer to peer feedback and evolving healthy, curious and humble teams. Tech Stack: Python, Javascript/Typescript, React/React Native, AWS, GraphQL, Snowflake, Airflow, DDD. This is an incredible opportunity for a Senior Software Engineer to join a unique company as they embark on a period of significant ...

working as a Data Engineer or in a similar data-focused engineering role Strong Python and SQL skills Experience building and orchestrating data pipelines (Airflow or similar tools) Hands-on work with cloud platforms (AWS and/or Azure) Solid understanding of databases, data warehouses, and data lakes Experience ...

modelling) ETL/ELT pipeline design, build, and orchestration Docker and containerisation Cloud environments (Azure, AWS, or GCP) Version control (Git) Bonus points for: Airflow, Prefect, dbt, or cloud-native orchestration CI/CD and deployment automation Data warehousing fundamentals (star schema, staging patterns) Plus: Ability to explain complex ...