1 to 25 of 55 Apache Airflow Jobs in London

maintain scalable, cloud-based data pipelines and architectures to support advanced analytics and machine learning initiatives. Develop robust ELT workflows using tools like dbt, Airflow, and SQL (PostgreSQL, MySQL) to transform raw data into high-quality, analytics-ready datasets. Collaborate with data scientists, analysts, and software engineers to ensure … however Azure exposure is essential. Data Warehousing: Proven expertise with Snowflake – schema design, performance tuning, data ingestion, and security. Workflow Orchestration: Production experience with Apache Airflow (Prefect, Dagster or similar), including authoring DAGs, scheduling workloads and monitoring pipeline execution. Data Modeling: Strong skills in dbt, including writing modular ...

similarly regulated industry. Strong coding skills in Python or Java (Spring Boot); React experience is a plus. Proficiency with modern data tools: Airflow, Spark, Kafka, dbt, Snowflake or similar. Experience with cloud platforms (AWS, Azure, GCP), containerization (Docker, Kubernetes), and CI/CD. Data Quality: Proven ability to validate ...

coding skills in Java (Spring Boot) for data engineering and services. Proven background working within financial services environments. Hands-on experience building and managing Apache Airflow/Spark and Hadoop clusters. Deep experience in big data engineering, including Hadoop, Kafka, and Spark-based data pipelines. Strong SQL/ ...

communicate clearly with technical and non-technical stakeholders Nice to Have Experience with dbt or other transformation frameworks Exposure to data orchestration tools (Airflow, etc.) Understanding of data governance, lineage, and metadata management Experience supporting BI tools (Power BI, Tableau, Looker, etc.) Python experience for data tooling or automation ...

communicate clearly with technical and non-technical stakeholders Nice to Have Experience with dbt or other transformation frameworks Exposure to data orchestration tools (Airflow, etc.) Understanding of data governance, lineage, and metadata management Experience supporting BI tools (Power BI, Tableau, Looker, etc.) Python experience for data tooling or automation ...

experience designing and building modern data platforms in AWS. ETL/Orchestration Expertise: Expertise in ETL/ELT design and data orchestration, specifically with Apache Airflow. SQL Mastery: Strong SQL skills with significant experience in query tuning and performance optimisation. Programming Proficiency: Proficiency in Python and Bash (for data ...

solving capabilities. · Excellent communication and documentation skills. · Ability to work independently and lead cross-functional teams. · Adaptability to rapidly evolving technological environments PREFERRED SKILLS · Airflow, dbt, Spark, Kafka, RabbitMQ, Redis. · Git, CI/CD pipelines. · Experience with data warehousing solutions such as Snowflake, Redshift, BigQuery, or Synapse. · Exposure ...

Data Engineering (not building pipelines but designing and building the framework) DevOps MLOps/LLMOps Often work with the following technologies : Azure, AWS, GCP Airflow, dbt, Databricks, Snowflake, etc. GitHub, Azure DevOps and related developer tooling and CI/CD platforms, Terraform or other Infra-as-Code MLflow, AzureML ...

using concepts, ETL/ELT pipelines and dimensional modelling Proficiency in advanced programming languages (Python/PySpark, SQL) Experience in data pipeline orchestration (e.g. Airflow, Data Factory) Familiarity with DevOps and CI/CD practices (Git, Azure DevOps etc) Ability to communicate technical concepts to both technical ...

schema). Version Control Practical experience with Git (branching, merging, pull requests). Preferred Qualifications (A Plus) Experience with a distributed computing framework like Apache Spark (using PySpark). Familiarity with cloud data services ( AWS S3/Redshift, Azure Data Lake/Synapse, or Google BigQuery/Cloud Storage … . Exposure to workflow orchestration tools ( Apache Airflow, Prefect, or Dagster ). Bachelor's degree in Computer Science, Engineering, Information Technology, or a related field. ...

Snowflake Production experience with dbt (mandatory) Strong SQL and Python programming skills Experience with Git-based workflows and DevOps practices Familiarity with orchestration tools (Airflow, Prefect) and ETL/ELT patterns Knowledge of cloud platforms (AWS, Azure) and security best practices ...

modular, reusable transformations using SQL and Python. Implement CI/CD pipelines and manage deployments via Git. Automate workflows using orchestration tools such as Airflow or dbt Cloud. Configure and optimise Snowflake warehouses for performance and cost efficiency. Required Skills & Experience 7+ years in data engineering roles. 3+ years ...

technologies (NoSQL, time-series databases) and deep competence in data modeling principles. Workflow Orchestration: Strong working knowledge of industry-standard data orchestration tools (e.g., Apache Airflow or Kubernetes). The Environment We are seeking a highly collaborative professional with strong problem-solving skills, capable of thriving ...

skills and experience with relational databases Python development experience and familiarity with frameworks (e.g. Flask, Django) Experience building and orchestrating ETL pipelines (e.g. Airflow, Luigi, Argo) Exposure to big data technologies (e.g. Spark, Kafka, Hadoop) Experience working in Agile environments Solid understanding of data architecture and data quality best ...

skills and experience with relational databases Python development experience and familiarity with frameworks (e.g. Flask, Django) Experience building and orchestrating ETL pipelines (e.g. Airflow, Luigi, Argo) Exposure to big data technologies (e.g. Spark, Kafka, Hadoop) Experience working in Agile environments Solid understanding of data architecture and data quality best ...

challenge the status quo and drive improvements. Confidence in collaborating across teams and markets to deliver impactful solutions. Nice-to-haves: Familiarity with Airflow , DBT , BigQuery , Redshift , Terraform , or Kubernetes . BENEFITS The successful Data Engineer will receive the following benefits: Salary up to £75,000 Flexible working arrangements ...

days/week from Central London office Strong problem-solving skills, attention to detail, and ability to work independently Experience with workflow orchestration (Prefect, Airflow, Dagster), cloud data warehouses (BigQuery, Snowflake), or backend Python frameworks (Django, FastAPI) is a plus Benefits 25 days annual leave plus bank holidays Meaningful ...

such as React or Next.js . Experience creating and maintaining APIs, working with relational/non‐relational databases. Familiarity with data orchestration tools (e.g. Airflow ) and data warehouse/lake design. Practical knowledge of AWS or Azure cloud environments. Strong understanding of Git , CI/CD , and modern deployment ...

Prometheus, Grafana, Alertmanager, Thanos; ELK; Jaeger; custom exporters Jenkins, GitLab, Sonatype Nexus, Terraform, Ansible, Argo CD AWS (primary), with GCP/Azure exposureDVC, MLFlow, Airflow, Dagster, CVAT, Weights &... ...

environment looks a bit like this 👇 👉 Python (pandas, numpy, scipy, PySpark) 👉 SQL 👉 AWS or GCP (BigQuery/RedShift/Snowflake) 👉 Spark/Databricks 👉 Airflow/Dagster ...

similar) and how they consume data Experience working with cloud environments (AWS, GCP) Familiarity with basic data quality testing and workflow orchestration concepts (e.g., Airflow) Experience working in Agile, cross-functional teams Python experience for automation or scripting would be advantageous ...

with hands on Snowflake experience* Proven dbt experience at scale* Strong dimensional modelling using Kimball principles* Python for data engineering tasks* Experience with Airflow or similar orchestration tools* Git based workflows* Familiarity with BI tools such as Tableau, Looker, or Power BI* Experience working in cloud environments including ...

technologies . You will be utilising and playing with tools across the entire data ecosystem. 🚀Essential tech skills include proficiency in Python, Spark, and Airflow , alongside experience with major data platforms such as Snowflake, Databricks, and BigQuery . 🚀Familiarity with data transformation and infrastructure as code tools like ...

have indepth technical knowledge of Python for data workflows, SQL and Data Modelling, DBT, Cloud platforms (AWS, GCP or Azure), Workflow Orchestration tools (e.g. Airflow, Dagster, Prefect, DBT Cloud), CI/CD (GitHub Actions, GitLab CI, CircleCI) You have experience of leading technical deployments, running customer projects, diagnosing issues ...

pipelines in a production environment. Exposure to cloud-based data platforms (AWS, GCP, Azure, or similar). Familiarity with data orchestration tools (e.g. Airflow or equivalent). Solid understanding of data warehousing concepts, schema design, and data modelling. Oscar Associates (UK) Limited is acting as an Employment Agency ...