1 to 25 of 54 Apache Airflow Jobs in London

governance, security, and regulatory compliance . Ability to work closely with technical and business teams to deliver impactful solutions. Nice-to-Haves: Orchestration tools (Airflow, dbt, Prefect) or streaming frameworks (Kafka, Kinesis). CI/CD experience and DevOps practices for data workflows. Exposure to data visualisation platforms (PowerBI ...

governance, security, and regulatory compliance . Ability to work closely with technical and business teams to deliver impactful solutions. Nice-to-Haves: Orchestration tools (Airflow, dbt, Prefect) or streaming frameworks (Kafka, Kinesis). CI/CD experience and DevOps practices for data workflows. Exposure to data visualisation platforms (PowerBI ...

architecture, modelling, pipelines) (please be clear on years and/or projects) Strong Python and SQL Experience with orchestration and transformation frameworks such as Airflow, dbt, Dagster , or similar Experience designing and delivering data platforms or lakehouse-style architectures Solid understanding of CI/CD for data engineering (e.g. ...

pipelines Proficiency with at least one cloud platform: Amazon Web Services (AWS) Microsoft Azure (Azure) Google Cloud Platform (GCP) Experience with orchestration tools (e.g., Apache Airflow) Experience with version control (e.g., Git) Solid understanding of data warehousing concepts and dimensional modeling To apply for this role please submit ...

industry. Strong full-stack experience with Python , Java (Spring Boot) , React , SQL, and relational databases. Hands-on experience with modern data engineering tools (e.g. Airflow, Spark, Kafka, dbt, Snowflake or similar). Experience with cloud platforms , containerisation (Docker/Kubernetes) , and CI/CD pipelines . Solid understanding ...

Location: London, UK (3 Days/Week Onsite) Type: Fixed Term Contract (6 Months) Mandatory Skills Needed : SQL ( Advanced) , AWS Cloud (Glue,Ethena,EMR), Airflow, Kafka/Kenesis ,Python, Terraform DBT & Snowflake (Basic) AWS Data Engineer Design, develop, and maintain data pipelines for collecting, transforming, and loading data into ...

Skills & Experience Proven experience as a Data Engineer , BI Engineer , or similar role. Strong proficiency in SQL , Python , and modern data engineering frameworks (e.g., Airflow, dbt, Spark, etc.). Experience with cloud platforms such as AWS, Azure, or GCP. Solid understanding of data warehousing and ETL/ELT architecture. ...

based data engineering, including Redshift, S3, IAM and related services Building and maintaining data pipelines using Python and SQL Orchestration tools such as Airflow Data modelling and warehouse architecture in modern cloud environments Git-based development and Infrastructure as Code (e.g. Terraform) Desirable experience includes exposure to regulated industries ...

working in fast-paced environments and able to contribute quickly without extensive onboarding. Nice to Have Experience with modern data stack tools (e.g., dbt, Airflow, etc.). Understanding of CI/CD pipelines and data infrastructure automation. Familiarity with data governance, security, and best practices in a cloud environment. ...

Ability to communicate complex technical concepts to non-technical stakeholders Nice to Have Experience with MLflow, DVC, SageMaker Workflow orchestration tools (Dagster/Airflow) Containerisation (Docker, ECS) Domain experience in transportation, airlines, operations, or network optimisation Qualifications Master’s degree in Data Science, ML, Operational Research OR 2+ years ...

detailed technical design. Hands-on experience configuring multi-node Hadoop clusters, including resource management, security, and performance tuning. Strong Big Data engineering background using Apache Airflow, Spark, dbt, Kafka, and Hadoop ecosystem tools. Knowledge of RDBMS systems (PostgreSQL, SQL Server) and familiarity with NoSQL/distributed databases such … MongoDB. Proven delivery of streaming pipelines and real-time data processing solutions. Improved job efficiency and reduced runtimes through Apache Spark optimisation and development. Some experience with containerisation (Docker, Kubernetes) and CI/CD pipelines. Delivered streaming pipelines and real-time data processing solutions. Experience replacing legacy ETL tools ...

detailed technical design. Hands-on experience configuring multi-node Hadoop clusters, including resource management, security, and performance tuning. Strong Big Data engineering background using Apache Airflow, Spark, dbt, Kafka, and Hadoop ecosystem tools. Knowledge of RDBMS systems (PostgreSQL, SQL Server) and familiarity with NoSQL/distributed databases such … MongoDB. Proven delivery of streaming pipelines and real-time data processing solutions. Improved job efficiency and reduced runtimes through Apache Spark optimisation and development. Some experience with containerisation (Docker, Kubernetes) and CI/CD pipelines. Delivered streaming pipelines and real-time data processing solutions. Experience replacing legacy ETL tools ...

working in fast-paced environments and able to contribute quickly without extensive onboarding. Nice to Have Experience with modern data stack tools (e.g., dbt, Airflow, etc.). Understanding of CI/CD pipelines and data infrastructure automation. Familiarity with data governance, security, and best practices in a cloud environment. ...

Data Engineer in a consultancy or complex delivery environment Deep, hands-on expertise with Snowflake Experience with the modern data stack, including dbt, Airflow (or similar) and Python Solid experience with AWS and/or Azure Strong SQL skills and performance optimisation experience Familiarity with CI/CD, DevOps ...

tasks, working with the Snowflake Data Marketplace, Performance Tuning and Knowledge of ETL/ELT pipelines and data integration tools (e.g., Informatica, Matillion, DBT, Airflow). Experience with APIs, stored procedures, and automation scripts is a plus. Strong analytical and problem-solving mindset. Clear communicator, able to work across ...

tooling Good scripting skills (Python/SQL/Bash) Strong understanding of AWS security, IAM, and networking Comfortable operating in Agile, delivery-led environments Apache Airflow for DAG Date-Led Platform/Cloud Engineering Key requirements: Active SC Clearance (essential) Permanent role Ideally you will be based around ...

experience withDWH platforms such as: Azure Data Warehouse/Synapse Microsoft Fabric AWS Redshift Snowflake Experience with ETL/ELT tools (SSIS, ADF, Matillion, Airflow, etc.) Desirable Skills Experience delivering solutions in secure government or defence environments Python proficiency for data engineering workflows CI/CD experience and data ...

Familiarity with cloud-based data tools and modern data ecosystems. Understanding of data modeling, warehousing concepts, and best practices Nice-to-Have Experience with Airflow, dbt, or similar orchestration/modeling tools. Experience with BI tools Knowledge of GCP services (Cloud Storage, Dataflow, Pub/Sub, etc.). Social ...

technical expertise with mentoring, guidance, and collaboration across teams. Core technologies include: Google Cloud Platform (BigQuery, Dataflow, Dataproc, Data Fusion, Data Streams, Cloud Functions, Airflow/Composer), SQL, Python, CI/CD tooling About you You'll have 10+ years experience in Data Engineering Experience operating in a Lead ...

data structures, performance, and optimisation strategies Familiarity with GCP and ecosystem knowledge: BigQuery, Composer, Dataproc, Cloud Run, Dataplex Hands-on experience with orchestration (like Airflow, Dagster, Databricks Workflows) and distributed processing in a cloud environment Experience with analytical data modelling (star and snowflake schemas), DWH, ETL/ELT patterns ...

grade data pipelines for multiple internal teams. Develops and optimises central data models supporting analytics, ML, and operational workloads. Works with Databricks, AWS, dbt, Airflow, and Looker as part of a modern data stack. Leads complex technical initiatives and improves efficiency through CI/CD, observability, and tooling. Ensures ...

Time-series data experience (market data, telemetry, pricing, events) Streaming exposure (Kafka/Event Hubs/Kinesis) Experience with Temporal (or similar orchestrators like Airflow/Dagster/Prefect) Any exposure to AI agents/automation tooling 👉 Apply ...

technologies . You will be utilising and playing with tools across the entire data ecosystem. 🚀Essential tech skills include proficiency in Python, Spark, and Airflow , alongside experience with major data platforms such as Snowflake, Databricks, and BigQuery . 🚀Familiarity with data transformation and infrastructure as code tools like ...

peer to peer feedback and evolving healthy, curious and humble teams. Tech Stack: Python, Javascript/Typescript, React/React Native, AWS, GraphQL, Snowflake, Airflow, DDD. This is an incredible opportunity for a Senior Software Engineer to join a unique company as they embark on a period of significant ...

where priorities evolve and things aren’t always fully specified Nice to Have (Not Essential) Experience with modern data platforms (e.g. Databricks, Snowflake, dbt, Airflow, Foundry) Some exposure to backend development or API design TypeScript experience Familiarity with enterprise systems (ERP, CRM, etc.) Exposure to AI-enabled systems , generative ...