20 of 20 Apache Airflow Jobs in the South East

Data Engineer 6 Month Contract Location: London (Onsite 4 days a week) Rate: £700 - £800 per day (INSIDE IR35) Skills: Python, Azure, Kubernetes, Apache Airflow, Apache Spark, DV Clearance We are looking to recruit a DV Cleared Data Engineer for a leading technology consultancy. This will … Work across Azure environments on the low side Support data engineering workloads within Kubernetes environments on the high side Develop and maintain workflows using Apache Airflow Work with Apache Spark for large-scale data processing Support the ongoing development and improvement of established data pipelines Help ensure ...

. Strong technical knowledge of SQL and Python . Experience with modern data technologies and tooling such as Microsoft Fabric, Power BI, Snowflake, dbt, Airflow, Databricks or Looker . Experience designing data pipelines, ETL/ELT processes and modern data models. Practical understanding of modern data architecture approaches, including ...

Kanban. Strong analytical and problem‐solving skills, with a focus on delivering high-quality solutions. Experience with modern data engineering tools such as Airflow, dbt, Dataform, or similar technologies. Excellent communication and stakeholder management skills, with the ability to explain technical concepts to non‐technical audiences. What we offer ...

Kanban. Strong analytical and problem-solving skills, with a focus on delivering high-quality solutions. Experience with modern data engineering tools such as Airflow, dbt, Dataform, or similar technologies. Excellent communication and stakeholder management skills, with the ability to explain technical concepts to non-technical audiences. Unfortunately ...

workflows Skills and experience: Passionfor building scalable, resilient cloud data platforms Strongexperience with Databricks or Snowflake on AWS ProvenPython skills, including Spark and Airflow expertise AdvancedSQL skills and end‐to‐end data modelling experience Experiencebuilding batch and real-time data integrations Hands-onCICD skills with Git, Jenkins or similar ...

production problems and taking responsibility for critical infrastructure. Startup Mindset: Proactive, resourceful and comfortable taking ownership in a fast-moving engineering environment. Bonus : with Apache Airflow, real-time data platforms, AI/ML infrastructure, LLM agents, GPU infrastructure or financial-market systems. Benefits An Ambitious, Collaborative Culture: Join ...

code Requirements: Must have strong, commercial AWS (contract) Data Engineering experience Must bring solid knowledge of data warehousing and modelling Experience with Airflow and infrastructure as a code (Terraform, CloudFormation) would be beneficial Must have experience with Redshift Nice to have: Experience with dbt/lake formation ...

cloud-native architectures, with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats ...

Pulumi, or equivalent technologies. Experience implementing monitoring and observability solutions using tools such as Prometheus, Grafana, or similar. Familiarity with orchestration platforms including Dagster, Airflow, Prefect, or related technologies. Desirable experience includes: Model serving infrastructure for real-time or batch inference workloads. Multi-tenant platform environments. Vector databases such ...

Shape the engineering approach across Google Cloud Platform (GCP) , working with technologies including BigQuery, Cloud Storage, Pub/Sub, Datastream and Cloud Composer/Airflow, alongside Terraform and infrastructure-as-code. Set and champion high engineering standards across data modelling, dbt (data build tool) transformations, testing, monitoring, documentation, deployment … BigQuery, and a good working knowledge of the wider Google Cloud Platform (GCP) ecosystem. Experience with modern data engineering tools and practices such as Airflow/Cloud Composer, dbt, Git/GitHub, CI/CD and Terraform or other infrastructure-as-code tooling . A solid understanding of data ...

live ML models. BAU Operator: High autonomy to own daily operations and deliver under tight deadlines. Cloud & BI Stack: Practical exposure to GCP (BigQuery), Airflow, or Looker . AI Tooling: Familiarity with Claude Code, OpenAI, Cursor , or agentic workflows to accelerate delivery. Compliance & Domain: Prior experience with FCA reporting ...

diligence work. Experience with MLOps or model lifecycle tooling such as MLflow, model registries, or feature stores. Experience with workflow orchestration tools such as Airflow, dbt, or Databricks Workflows. Experience with data product or data mesh operating models. Knowledge of European banking or EU regulatory frameworks. Experience with Teradata ...

platform and expanding their engineering capability. The Role Own and develop the MLOps infrastructure supporting AI services. Build and maintain orchestration pipelines using Dagster, Airflow, or Prefect. Deploy and manage ML workloads on Kubernetes and cloud platforms. Implement CI/CD, observability, and monitoring across production systems. Develop infrastructure ...

experimentation, statistical analysis and performance measurement. Ability to influence senior stakeholders and translate analytics into commercial outcomes. Experience with tools such as BigQuery, dbt, Airflow, Git, Tableau or Looker is advantageous. Python experience is beneficial. What They Offer Up to £100,000 base salary plus bonus and benefits. Hybrid ...

codebase and pipelines Hands-on experience maintaining production ML models: refresh, retrain, monitor, and troubleshoot, rather than building from scratch Working knowledge of Looker, Airflow and Google Cloud Advanced degree in a quantitative discipline (Computer Science, Mathematics, Statistics, Physics, or similar), or equivalent experience Benefits Hybrid working (3 days ...

PostgreSQL. Some exposure to networked services and APIs, such as gRPC or REST. Exposure to schedulers, job queues or workflow engines - Celery, Airflow, Temporal, Slurm or similar. Comfort working on Linux, and debugging a service that is already running. Good communication skills both within software and across disciplines. ...

machines, timeouts, leases, retries, and the failure modes that only show up under load. Experience with a scheduler, workflow engine or job queue - Temporal, Airflow, Celery, Slurm, or one you built - or with event‐driven systems that react to changes in state. A habit of settling design questions ...

reach for determinism when chasing a timing or interleaving bug, rather than for more logging. Depth in the scheduling and workflow landscape - Temporal, Hatchet, Airflow, Slurm, Postgres-backed queues - and the judgement to borrow from it rather than reinvent. A track record of pinning down failures that others could ...

that powers a suite of AI and machine learning services. Design, build, and maintain production-grade ML orchestration pipelines using tools such as Dagster, Airflow, Prefect, or similar technologies. Deploy, monitor, and scale machine learning models and LLM-powered solutions in production environments. Build and maintain cloud-native infrastructure … have: Strong commercial experience in Python software engineering. Experience deploying and managing machine learning infrastructure in production. Knowledge of orchestration tools such as Dagster, Airflow, Prefect, or similar. Hands-on experience with Kubernetes and containerised workloads. Expertise in Infrastructure as Code using Terraform, Bicep, Pulumi, or comparable technologies. Experience ...

Inside IR35 Location: London - Hybrid, 3 days per week onsite Duration: 6 months The role will focus on GCP, BigQuery, Python, Airflow/Cloud Composer, dbt, Terraform and CI/CD, working closely with architects, engineers, product and governance teams to deliver secure, scalable data solutions. Key experience … BigQuery Python, Airflow & dbt Terraform & CI/CD Data platforms/lakehouse environments Kubernetes AWS alongside GCP Strong stakeholder management If this sounds relevant to you, please drop me a message or send your CV Thanks, Callum ...