176 to 200 of 235 Apache Airflow Jobs in the UK

data and analytics programmes, ensuring alignment with business goals, timelines, and quality standards. Planning, tracking, and managing complex technical dependencies across Snowflake, dbt, Airflow, and GitHub‐based development workflows. Collaborating closely with data engineering and analytics teams to support Snowflake architecture, performance optimisation, and cost management. Overseeing dbt project …/experience: Proven experience in a Technical Program Manager, Technical Project Manager, or similar role. Strong hands‐on understanding of Snowflake, dbt, GitHub, and Airflow in a production environment. Demonstrable experience delivering data engineering or analytics platform programmes. Solid understanding of modern data stacks, ETL pipelines, and cloud‐based ...

across the data engineering domain. Engage with data engineering technical detail when needed, understanding the data platform landscape including data ingestion, transformation (DBT), orchestration (Airflow), streaming (Kafka), and data warehousing (Snowflake), and able to articulate technical choices and trade-offs to different stakeholders. Work at both conceptual and detailed … despite complexity in data requirements and priorities. What I need to know: Strong understanding of data engineering technologies including data ingestion, transformation (DBT), orchestration (Airflow/Astronomer). Experience with cloud data platforms (AWS data services, Azure data services) including cost management and optimization Knowledge of data quality frameworks ...

preferred Mastery of Python, SQL, and Spark – and strong opinions on how they should be used in production data pipelines Deep experience with dbt, Airflow, or equivalent orchestration – including how to build pipelines that are testable, observable, and operationally sound Strong understanding of data quality engineering – not just validation … leadership responsibility Primary cloud AWS – Glue, Lambda, Athena, S3, Redshift, RDS, Kinesis, MSK, Lake Formation, Step Functions, MWAA Streaming Languages Python, SQL, Spark dbt, Airflow/AWS MWAA, AWS Glue Platforms Databricks, Snowflake, Amazon Redshift Great Expectations, dbt tests, reconciliation frameworks, data observability tooling IaC/DevOps Terraform ...

preferred Mastery of Python, SQL, and Spark – and strong opinions on how they should be used in production data pipelines Deep experience with dbt, Airflow, or equivalent orchestration – including how to build pipelines that are testable, observable, and operationally sound Strong understanding of data quality engineering – not just validation … leadership responsibility Primary cloud AWS – Glue, Lambda, Athena, S3, Redshift, RDS, Kinesis, MSK, Lake Formation, Step Functions, MWAA Streaming Languages Python, SQL, Spark dbt, Airflow/AWS MWAA, AWS Glue Platforms Databricks, Snowflake, Amazon Redshift Great Expectations, dbt tests, reconciliation frameworks, data observability tooling IaC/DevOps Terraform ...

multiple teams. In this contract role, expect to define standards, build reusable data movement frameworks and drive best practices with AWS Glue, Spark and Airflow in a regulated environment. London-based with on-site presence three days a week. #J-18808-Ljbffr ...

software in on-premise, edge or customer-hosted environments. Monitoring, logging and production observability. Desirable Skills Kubernetes. MLflow, Weights & Biases or similar MLOps tooling. Airflow, Prefect, Kubeflow or other workflow orchestration tools. Time-series or telemetry data. Distributed training workloads. Industrial, IoT or edge computing environments. Why Join? Work ...

staking, or other on-chain systems. Full-stack development with Python (FastAPI). Strong relational data modeling fundamentals. GitOps and Kubernetes. Job orchestration systems (Airflow, Prefect, etc.). For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice ...

advanced proficiency in Python with extensive experience building production applications and services Experience with modern cluster compute technologies (e.g., Kubernetes, Spark, Hadoop Yarn, Ray, Airflow) Experience leading work in a formal software development environment in collaboration with other engineers Strong experience building platforms on the Linux operating system Self ...

clear, performant, and reliable queries to transform large complex datasets) You have experience building and owning automated data pipelines with tools like DBT/Airflow Experimentation & Statistics: Experience running A/B tests and analysing the results to drive product decisions Ownership: You enjoy working independently in a fast ...

Advanced proficiency in Python, with extensive professional experience in building production applications and services Experience of modern cluster compute - Kubernetes, Spark, Hadoop Yarn, Ray, Airflow are some of the technologies we use Experience leading work in a formal software development environment alongside other engineers in a collaborative manner Strong ...

technical design reviews. Must Haves (Hard Skills) Data Lakehouse Mastery: Proven experience building data infrastructure using Trino/Presto and open table formats like Apache Iceberg or Delta Lake. Advanced Data Modeling: Deep expertise in Medallion Architecture patterns, Star Schemas, and processing semi‐structured financial data logs (JSON/… Avro). Robust ETL/ELT Orchestration: Expert proficiency in data transformation engines (such as dbt) and scheduling tools (such as Airflow). Stream/Message Queues: Solid understanding of event‐driven architectures and streaming processing via Apache Kafka. Strong Programming Foundations: Proficiency in Python ...

technical design reviews. Must Haves (Hard Skills) Data Lakehouse Mastery: Proven experience building data infrastructure using Trino/Presto and open table formats like Apache Iceberg or Delta Lake. Advanced Data Modeling: Deep expertise in Medallion Architecture patterns, Star Schemas, and processing semi-structured financial data logs (JSON/… Avro). Robust ETL/ELT Orchestration: Expert proficiency in data transformation engines (such as dbt) and scheduling tools (such as Airflow). Stream/Message Queues: Solid understanding of event-driven architectures and streaming processing via Apache Kafka. Strong Programming Foundations: Proficiency in Python ...

/ML, data engineering) Experience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal) Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governance Practical cloud-native experience (AWS, Azure … hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with AI evaluation and observability practices ...

mechanics (order books vs AMMs, fee structures, market manipulation/anti-manipulation signals). Proficiency with SQL; experience with streaming systems (Kafka), workflow orchestration (Airflow), and cloud (AWS/GCP/Azure). Experience with Bayesian methods, probabilistic programming (Stan/PyMC), or ensemble methods. Familiarity with rigorous experimentation … online/offline evaluation, data leakage prevention, and model governance. Tech Stack Python, SQL, pandas/numpy/scipy, PyTorch/sklearn Airflow/dbt, Kafka (or equivalents), Postgres/BigQuery Docker, Kubernetes (optional), CI/CD (GitHub Actions) Observability: Prometheus/Grafana, OpenTelemetry (or equivalents) Locations and Right ...

responsibilities: Building and maintaining data pipelines and services under the guidance of senior engineers. Working with technologies such as Python, SQL, Spark, and Airflow to process large-scale data. Using AI-assisted development tools such as Claude Code, Copilot, or Codex as part of your workflow. Contributing … Basic understanding of data engineering concepts including batch processing and relational databases. Some hands-on experience with data pipeline tools; exposure to Spark or Airflow is a plus. Familiarity with cloud platforms (preferably AWS); production experience is not required. A collaborative mindset and willingness to learn and communicate openly ...

Title: Analytics Engineer Location: Manchester (Hybrid, 2-3 days per week in the office) Salary: £60,000 - £80,000 DOE Industry: Financial Services The Opportunity I'm currently working with a growing financial services organisation ...

recommendations, ranking and personalization across Bumble products. You will build production-grade models with PyTorch, craft end-to-end ML pipelines using Spark and Airflow, and deploy AI Agents based on Large Language Models, mentoring senior contributors and shaping responsible AI practices. #J-18808-Ljbffr ...

deliver scalable, production-grade Python services and APIs for research and operations. The role emphasizes modern cluster compute technologies (Kubernetes, Spark, Hadoop Yarn, Ray, Airflow) and strong proficiency in Python. A background in mathematics or computing is advantageous. #J-18808-Ljbffr ...

improve go-to-market pricing and revenue management, collaborating with pricing, monetization, and engineering teams. The role offers exposure to Vertex AI, BigQuery, and Airflow, plus generous benefits and a culture built on trust and inclusion. #J-18808-Ljbffr ...

engineers across the firm. Ideal candidates have deep Python expertise, hands-on experience with modern cluster compute technologies (Kubernetes, Spark, Hadoop/Yarn, Ray, Airflow) and a track record of delivering scalable infrastructure in a multi-user #J-18808-Ljbffr ...

topologies. Exposure to MLOps tooling and AI platform layers: experiment tracking (MLflow, W'B), model serving frameworks (Triton, vLLM), and pipeline orchestration (Kubeflow, Airflow). Familiarity with InfiniBand and high-performance networking as it relates to distributed training performance — sufficient to engage credibly with network and infrastructure teams ...

evaluation harness in production. Familiarity with retrieval/IR systems and the engineering of large-context pipelines. Distributed‐systems background — workflow engines (Temporal/Airflow/Prefect), queues, and observability stacks (OpenTelemetry, Datadog, Honeycomb). Open‐source contributions to agentic frameworks, eval tooling, or workflow orchestration. Why Join ...

experience with cloud platforms (AWS, Azure, GCP) Experience in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g. Apache NiFi, Airflow) Practical experience with NLP and/or Search techniques Prompt engineering, and LLMs for enterprise‐scale applications. You have team lead ...

cloud platforms (AWS, Azure, GCP). Well‐versed in Python, with experience in frameworks like Tensorflow, PyTorch, and tools for building ETL pipelines (e.g., Apache NiFi, Airflow). Strong collaboration skills with the ability to work across engineering, research, and product teams across multiple time zones. You have ...

Solutions Architect to join our field team and deliver world-class technical solutions. You will work with customers to implement Astronomer products, focusing on Apache Airflow, and guide complex deployments at scale. In this hands-on role, you will collaborate with Sales and Product, solve intricate architecture challenges ...