26 to 50 of 58 Apache Airflow Jobs in Central London

tech firm Strong hands-on expertise in Python and modern ETL frameworks Experience designing and maintaining cloud-based data pipelines (e.g. AWS, Airflow, Snowflake) Deep understanding of data modelling, validation, and pipeline resilience Familiarity with financial or alternative datasets preferred ...

growth, including Agentic development. Technology/Skills requirements SQL ELT/ETL Data Modelling dbt models Version control (Git) Data processing and orchestration using Airflow or similar Google BigQuery or Redshift Cloud based services: GCP (preferred) or AWS Agentic development frameworks Automations About You 4 years+ experience ...

have Google Cloud Platform (GCP) – must-have, ie BigQuery, Cloud Run, and Cloud Storage dbt – strong advantage, with scope to shape best practice Airflow/Composer – highly desirable for orchestration CI/CD – GitHub Actions (or similar) Salary: £80,000 - £85,000 + Equity, PMI, Strong pension, clear progression ...

similar distributed compute frameworks. Experience managing cloud infrastructure, Kubernetes, and CI/CD pipelines. Familiar with dataset versioning, orchestration, and experiment tracking (DVC, MLflow, Airflow). Ideally, we are looking for someone with prior experience working at an early-stage start-up. ...

system design A pragmatic mindset: balancing engineering quality with delivery Nice to Have Experience with digital media, content workflows, or YouTube APIs Familiarity with Airflow, Composer, Dagster, or Prefect ML/AI awareness (classification, embeddings, tagging workflows) Experience with high-throughput scheduling or rules engines Please apply ...

integrate AI/LLM components into production systems. Role Breakdown: 50% Backend Engineering: FastAPI, Flask, Node.js, CI/CD 30% Data Engineering: ETL, DBT, Airflow 20% AI/LLM Integration: LangChain, RAG pipelines, orchestration Key Responsibilities: Design and build backend services to support AI agent deployment Develop scalable data ...

roadmaps, proposals, and PoC preparations, collaborating with Data/AI Engineers on full lifecycle deployments Build & deploy scalable solutions: create data pipelines (ETL/Airflow), MLOps (Docker/AWS/Azure/GCP), and interactive dashboards (Tableau/Power BI) Drive client impact – from EDA and model validation ...

cloud-native or AI-oriented environments. Strong Python skills for data pipeline and service development. Proficiency with AWS (Lambda, S3, Glue) and orchestration tools (Airflow, Dagster, etc.). Familiarity with relational and vector databases (PostgreSQL, Pinecone, MySQL). Understanding of API development and event-driven architectures. A mindset that ...

technical and non-technical stakeholders. Desirable Experience working in high-growth or scale-up environments. Exposure to BI tools. Familiarity with orchestration tools (e.g. Airflow, Dagster). Interest in data quality, governance, and metrics standardisation. Experience working in logistics, supply chain, marketplace, transportation, or operationally complex environments is highly ...

Code skills. Experience designing and delivering modern IT/cloud architectures. Knowledge of modern data/ML platforms (e.g., Databricks, Azure ML, Synapse, MLflow, Airflow). Desirable Disaster recovery and cloud security best practices. Experience in research, life sciences, or health data environments. Data sharing agreements and DPIAs. Snowflake ...

tools, helping the business make confident, data-driven decisions. Why this role is exciting: • Work in a modern data environment with tools like dbt, Airflow, Python, AWS, Databricks and Looker/BI platforms • Build scalable data models, pipelines and self-serve analytics solutions • Automate processes and improve operational efficiency ...

time data processing Experience building data pipelines on-prem Define and implement standards for data modelling, orchestration, version control, and data quality (e.g. dbt, Airflow) Embed security best practices, including RBAC, encryption, masking, and regulatory compliance Collaborate with technical and non-technical stakeholders to define data requirements, SLAs ...

Develop and maintain secure data access controls, including RBAC, token policies, and anonymisation mechanisms Support batch and real-time data flows using tools like Airflow, Kafka, Spark, and Terraform Monitor cloud platform performance and implement cost-control measures while improving reliability Collaborate with product, engineering, and governance teams ...

infrastructure as code exposure Familiarity with Docker, Terraform, Kubernetes, and modern cloud services Working knowledge of MLOps concepts and tooling (for example MLflow, Airflow) Comfortable enabling others: documentation, training, workshops, and hands on support Desirable LLMops exposure (plus) Experience using AI agent tools in real engineering workflows (for example ...

start-ups or high-growth scale-ups • Strong technical grounding across Node.js, Typescript, React, MySQL and cloud-native architectures (AWS, ECS/K8s, Kafka, Airflow) • Comfortable driving Agile and DevOps culture, and setting high standards for clean, testable, secure code in fast-moving environments If you want ...

start-ups or high-growth scale-ups • Strong technical grounding across Node.js, Typescript, React, MySQL and cloud-native architectures (AWS, ECS/K8s, Kafka, Airflow) • Comfortable driving Agile and DevOps culture, and setting high standards for clean, testable, secure code in fast-moving environments If you want ...

resilience and data residency Confidence working with senior stakeholders to make pragmatic, commercial platform decisions Tech environment includes: AWS, Azure, Kubernetes (EKS), ECS, Terraform, Airflow, EFK/Prometheus/Grafana, MySQL/PostgreSQL/Redis/Elasticsearch/Kafka, GitHub Actions, plus services written in Python, Node.js and Go. ...

resilience and data residency Confidence working with senior stakeholders to make pragmatic, commercial platform decisions Tech environment includes: AWS, Azure, Kubernetes (EKS), ECS, Terraform, Airflow, EFK/Prometheus/Grafana, MySQL/PostgreSQL/Redis/Elasticsearch/Kafka, GitHub Actions, plus services written in Python, Node.js and Go. ...

Design and implement model registries, versioning systems, and experiment tracking to ensure full reproducibility of all model releases. Deploy ML workflows using tools like Airflow or similar, managing dependencies from data ingestion through model deployment and serving. Instrument comprehensive monitoring for model performance, data drift, prediction quality, and system ...

Consumer Finance or Retail Banking is a plus. Experience with cloud data environments (e.g. Redshift, Azure) welcomed. Knowledge of automation tools such as Airflow is beneficial. Excellent communication skills, confident engaging with senior stakeholders. Proactive, results-driven, and comfortable working in a fast-paced environment. ...

Linux environment. - Experience in building automation tools and managing system configurations. - Knowledge of C++, KDB/q, and experience with technologies like Slurm, Airflow, Kafka, or AMPS. - Background in enhancing system stability, scalability, and performance while conducting root cause analyses to resolve incidents efficiently. - Observability skillset, monitoring and analysis ...

third-party data from APIs into scalable storage layers • Developing dbt transformations and ELT workflows for analytics and machine learning • Orchestrating workloads using Airflow and modern CI/CD practices • Supporting model execution environments, including Azure ML Studio (experience not required) What you’ll be doing • Designing and building … scalable data pipelines in a modern Azure environment • Developing modular, production-grade ELT workflows (dbt, Airflow, SQL, Python) • Modelling data for analytics, BI, forecasting, and machine learning use cases • Optimising data architectures for performance, cost, and reliability • Working closely with data scientists, software engineers, and investment teams • Troubleshooting ...

e.g. AWS, GCP, Azure), infrastructure-as-code, and scalable architecture patterns Hands-on experience with SQL, Python, and modern data integration frameworks such as Apache Kafka, airflow or dbt Proven ability to deliver enterprise-grade customer data solutions, including customer data platforms and or API based integration Excellent ...

schemas to support business requirements Develop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL) You have framework experience within either Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving ...

job. You will be taking ownership of a stack built on: Ingestion: Weld Warehousing & ML: BigQuery and Vertex AI Transformation: dbt Orchestration: Airflow BI: Looker Studio What This Role Involves This is a Full Stack Analytics Engineering role. You will own the lifecycle of data from the API source … Meta, Google Ads, Shopify, GA4, and other sources into BigQuery. Orchestration: You will manage the scheduling, dependencies, and monitoring of our data workflows using Airflow , ensuring data arrives on time and in the correct sequence. Custom Ingestion: You will write custom Python scripts/extractors to fetch data where ...