51 to 75 of 86 Apache Airflow Jobs in the UK

cloud-native architectures, with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats ...

Data Engineer within GCP. Experience building production-grade data pipelines and platforms. Expertise with technologies such as BigQuery, Dataflow, Cloud Composer/Airflow, Pub/Sub and Cloud Storage. Strong SQL and data engineering fundamentals. An engineering mindset and a passion for solving problems. Confidence working independently while remaining ...

operate across engineering, product, and business functions Desirable Experience LLM applications and document-processing platforms Retrieval-Augmented Generation (RAG) React and Next.js Snowflake Airflow Data engineering and ETL pipelines Kubernetes and cloud-native infrastructure Supply chain, procurement, or operational systems Ideal Background Suitable candidates may come from: Product ...

with product squads and business stakeholders Develop robust, production-quality software in Python and build large-scale data workflows using orchestration platforms such as Airflow, while driving technical execution across complex initiatives. Use expertise in cloud computing (AWS and Azure) to build and optimize AI infrastructure Champion ML governance ...

modelling, transformation, and visualization (eg Python Dash)Experience with software development (modularisation, versioning eg with GIT, tests, understandable code), with containerization (Docker) and orchestration (Airflow)Habit of starting simple first to get buy-in early and iterate on feedback to add valueCuriosity & desire to understand the worldA plus: existing ...

field Production ML engineering excellence – Deep understanding of ML operations, including containerization (Docker, Kubernetes/AWS EKS), cloud platforms (AWS/Azure), workflow orchestration (Airflow), automated testing for ML systems, and API development for model deployment Technical leadership and influence – Track record of mentoring and coaching engineers, driving ...

field Production ML engineering excellence – Deep understanding of ML operations, including containerization (Docker, Kubernetes/AWS EKS), cloud platforms (AWS/Azure), workflow orchestration (Airflow), automated testing for ML systems, and API development for model deployment Technical leadership and influence – Track record of mentoring and coaching engineers, driving ...

models, schemas to support business requirementsDevelop and maintain data ingestion and processing systems using various tools and technologies, such as SQL, NoSQL, ETL, Luigi, Airflow, Argo, etc. Implement data storage solutions using different types of databases, such as relational, non-relational, or cloud-based. Working collaboratively with the client …/Azure SQL, PostgreSQL)You have framework experience within either Flask, Tornado or Django, DockerExperience working with ETL pipelines is desirable e.g. Luigi, Airflow or ArgoExperience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data qualityPreparing ...

quality Self-driven, motivated work ethic Energetic and enthusiastic personality What makes you stand out: IMC or another investment management qualification Experience with Apache Airflow Experience with Databricks Experience with asset management data Passion for design and architecture Why join our team: We help you be your best ...

Python, APIs, and distributed systems. Good understanding of MLOps tooling, including CI/CD, monitoring, and pipeline management. Experience with tools such as MLflow, Airflow, Evidently, DVC, Docker, or Kubernetes. Experience working within AWS, GCP, or Azure environments. Previous mentoring or leadership experience with an interest in developing management ...

patterns, and integration with enterprise systems. Data Engineering & PlatformsAbility to build and maintain data pipelines ingestion, transformation, serving, and monitoring using tools such as Airflow, Kafka, dbt, or equivalents. Working familiarity with modern data platforms (Snowflake, Databricks, BigQuery, or similar), including data modeling and performance fundamentals. Strong debugging instincts ...

Companys hybrid work from home policy. Qualifications Commercial GCP background including BigQuery, Dataform and GCS Knowledge of Data Lake technologies such as Airflow, Kafka, Flink and Python Relational set-based processing through complex SQL queries Proven ability to leverage AI within a production workflow Background working within a Data ...

things done. Good to have: Experience in Financial Crime and/or Customer Onboarding domain Knowledge of the fintech ecosystem Knowledge of utilities like Airflow/Github/DBT Experience with user-facing products and data Additional Information What do we offer: Salary: £75,000 - £115,000 Company Restricted ...

working on the most impactful opportunities NICE TO HAVE BUT NOT ESSENTIAL Experience delivering high quality data and self-serve analytics using tools like Airflow, dbt and Looker. Practically used + applied AI/machine learning techniques and built predictive models to get deeper insights Knowledge of the FinTech ...

develop. On the data pipeline side, we value awareness across both established tools such as SSIS/SSRS and newer approaches such as dbt, Airflow, or AWS Glue — dbt experience is essential, with the others a strong plus. DesirablePrevious experience in working in cloud environments (AWS preferred … definite plus. Familiarity with legacy SSIS/SSRS, useful for maintaining existing packages, as well as newer orchestration/ingestion tools such as Airflow or AWS Glue. Exposure to cloud data warehouse/data fabric platforms such as Snowflake or Databricks. A good understanding of financial markets. Background ...

deploying, monitoring and maintaining ML systems at scale Ability to balance hands-on engineering with people management responsibilities Experience with AWS, Snowflake, MLflow, Airflow or similar tooling would be advantageous SALARY AND BENEFITS £110,000 - £125,000 base salary Annual bonus Hybrid working model Private medical insurance Enhanced family ...

/security implications. You might also have: Experience with Kubernetes for cloud-burst or elastic compute scenarios. Familiarity with workflow orchestration tools (e.g. Prefect, Airflow, Dagster) — not required to design flows, but useful for understanding deployment/runtime needs. Exposure to high-performance/scientific/technical computing environments ...

than one-off scripts or analysis tooling. Strong Python development skills, with experience building performant, maintainable applications. Experience with workflow orchestration tools such as Airflow, Dagster, or Prefect. Understanding of retries, dependency management, idempotency, backfills, and operational recovery. Experience with analytical or columnar databases such as ClickHouse or similar ...

clear, performant, and reliable queries to transform large complex datasets) You have experience building and owning automated data pipelines with tools like DBT/Airflow Experimentation & Statistics: Experience running A/B tests and analysing the results to drive product decisions Ownership: You enjoy working independently in a fast ...

maintain standardized data models, schemas, and metadata that make commodities datasets easy to understand, discover, and reuse. Schedule, monitor, and manage data pipelines using Airflow or similar workflow orchestration tools to ensure reliable, timely data delivery. Implement validation, reconciliation, and anomaly-detection controls to maintain data completeness, accuracy … building and maintaining ETL pipelines. Strong Python and SQL skills, including experience working with large datasets and complex data transformations. Hands-on experience with Airflow or comparable workflow orchestration tools. Experience with Git, CI/CD pipelines such as GitHub Actions, and automated testing frameworks such as PyTest. Strong ...

Engineering, or MLOps environments Experience building, deploying, and supporting machine learning solutions at scale Experience working with AWS and ML tooling such as MLflow, Airflow, Metaflow, or Snowflake Ability to balance technical leadership, strategic thinking, and hands-on engineering contribution SALARY AND BENEFITS Up to £125,000 base salary ...

experience with cloud platforms (AWS, Azure, GCP)Experience in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow)Practical experience with NLP and/or Search techniquesPrompt engineering, and LLMs for enterprise-scale applications. You have team lead experienceStrong ...

integration failures, data inconsistencies, and production issues Key Skills • Python & SQL (production-grade pipeline development) • REST APIs, webhooks, OAuth, event-driven architecture • Orchestration tools: Airflow, Prefect, or Dagster • Cloud platforms: AWS, GCP, or Azure • Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills • Vector databases and embedding pipelines • Real-time … robust, production-grade data pipelines from scratch. • Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. • Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). • Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

business uses data and technology to optimise pharmaceutical sourcing and supply chains. THE ROLE Build and enhance scalable data pipelines using Snowflake, dbt, Airflow and Python. Improve platform reliability, testing, monitoring and CI/CD processes. Partner with Analytics, Data Science and Engineering teams to deliver data solutions. Engage … drive business value. YOUR SKILLS AND EXPERIENCE 4+ years' experience in Data Engineering with Strong SQL and Python skills Experience with Snowflake, dbt and Airflow (or similar tools) Exposure to CI/CD, automated testing and cloud data platforms Strong communication and stakeholder management skills Databricks/PySpark experience ...

reach for determinism when chasing a timing or interleaving bug, rather than for more logging. Depth in the scheduling and workflow landscape - Temporal, Hatchet, Airflow, Slurm, Postgres-backed queues - and the judgement to borrow from it rather than reinvent. A track record of pinning down failures that others could ...