101 to 125 of 141 Apache Airflow Jobs in London

driving engineering best practices. Ability to influence technical direction and architectural decisions. Comfortable remaining hands-on while operating in a leadership capacity. Desirable Skills Airflow Databricks AI-focused development experience Semantic layer experience Reasonable Adjustments: Respect and equality are core values to us. We are proud of the diverse ...

platform engineer and presenting a business case to a C‐minus‐1. Preferred, not required Experience with the modern data stack: Snowflake, Databricks, dbt, Airflow, BigQuery, Spark, or similar. Data observability, data quality, or AI observability experience is a strong plus. Familiarity with MEDDPICC, Command of the Message ...

Service Meshes etc.) Experience with configuration management tools (Ansible/Puppet/Kapitan/Terraform) Experience with distributed data platforms (Kafka/Flink/Airflow) Comfortable using cloud native and containerisation technologies (Kubernetes/Docker) Good Linux systems knowledge (experience with RHEL desirable) Broad knowledge across network technologies, server ...

/CD experience. Highly Desirable Familiarity with OneTick, PostgreSQL, ClickHouse, Snowflake, Databricks, kdb+ or similar. Order book reconstruction Tick-to-bar aggregation Airflow, Dagster, Prefect or MLflow Experience supporting quantitative research or trading systems Multi-asset market data (Equities, Futures, FX, Options, ETFs, CFDs) What We're Looking … market data engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience in MLOps is advantageous but not essential. We welcome ...

including model deployment, monitoring, evaluation, and lifecycle management Strong hands‐on understanding of modern data and AI tooling, ideally including SQL, Python, Snowflake, dbt, Airflow, Metabase, Tableau, and AWS‐based data services Experience with data ingestion, streaming, and change data capture tooling, such as Fivetran, Estuary, or similar platforms … standards, and enablement for analytics engineers Ability to oversee API‐based and batch data extraction processes, particularly those built in Python and orchestrated using Airflow Understanding of machine learning tooling and lifecycle management, ideally including DVC, MLflow, Evidently AI, or equivalent platforms for experiment tracking, versioning, monitoring, and evaluation ...

data and analytics programmes, ensuring alignment with business goals, timelines, and quality standards. Planning, tracking, and managing complex technical dependencies across Snowflake, dbt, Airflow, and GitHub‐based development workflows. Collaborating closely with data engineering and analytics teams to support Snowflake architecture, performance optimisation, and cost management. Overseeing dbt project …/experience: Proven experience in a Technical Program Manager, Technical Project Manager, or similar role. Strong hands‐on understanding of Snowflake, dbt, GitHub, and Airflow in a production environment. Demonstrable experience delivering data engineering or analytics platform programmes. Solid understanding of modern data stacks, ETL pipelines, and cloud‐based ...

across the data engineering domain. Engage with data engineering technical detail when needed, understanding the data platform landscape including data ingestion, transformation (DBT), orchestration (Airflow), streaming (Kafka), and data warehousing (Snowflake), and able to articulate technical choices and trade-offs to different stakeholders. Work at both conceptual and detailed … despite complexity in data requirements and priorities. What I need to know: Strong understanding of data engineering technologies including data ingestion, transformation (DBT), orchestration (Airflow/Astronomer). Experience with cloud data platforms (AWS data services, Azure data services) including cost management and optimization Knowledge of data quality frameworks ...

multiple teams. In this contract role, expect to define standards, build reusable data movement frameworks and drive best practices with AWS Glue, Spark and Airflow in a regulated environment. London-based with on-site presence three days a week. #J-18808-Ljbffr ...

software in on-premise, edge or customer-hosted environments. Monitoring, logging and production observability. Desirable Skills Kubernetes. MLflow, Weights & Biases or similar MLOps tooling. Airflow, Prefect, Kubeflow or other workflow orchestration tools. Time-series or telemetry data. Distributed training workloads. Industrial, IoT or edge computing environments. Why Join? Work ...

building and leading a team of engineers working in a related system. Experience with build, release and deployment infrastructure technologies such as Spinnaker, Jenkins, Airflow, Docker, Kubernetes, Terraform, Bazel, etc. Ability to attract, hire, and coach engineers who meet the Databricks hiring standards – can up‐level existing team ...

contracts, staking, or other on-chain systems Full-stack development with Python (FastAPI) Strong relational data modeling fundamentals GitOps and Kubernetes Job orchestration systems (Airflow, Prefect, etc.) For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice at https ...

advanced proficiency in Python with extensive experience building production applications and services Experience with modern cluster compute technologies (e.g., Kubernetes, Spark, Hadoop Yarn, Ray, Airflow) Experience leading work in a formal software development environment in collaboration with other engineers Strong experience building platforms on the Linux operating system Self ...

clear, performant, and reliable queries to transform large complex datasets) You have experience building and owning automated data pipelines with tools like DBT/Airflow Experimentation & Statistics: Experience running A/B tests and analysing the results to drive product decisions Ownership: You enjoy working independently in a fast ...

Advanced proficiency in Python, with extensive professional experience in building production applications and services Experience of modern cluster compute - Kubernetes, Spark, Hadoop Yarn, Ray, Airflow are some of the technologies we use Experience leading work in a formal software development environment alongside other engineers in a collaborative manner Strong ...

Advanced proficiency in Python, with extensive professional experience in building production applications and services Experience of modern cluster compute - Kubernetes, Spark, Hadoop Yarn, Ray, Airflow are some of the technologies we use Experience leading work in a formal software development environment alongside other engineers in a collaborative manner Strong ...

technical design reviews. Must Haves (Hard Skills) Data Lakehouse Mastery: Proven experience building data infrastructure using Trino/Presto and open table formats like Apache Iceberg or Delta Lake. Advanced Data Modeling: Deep expertise in Medallion Architecture patterns, Star Schemas, and processing semi‐structured financial data logs (JSON/… Avro). Robust ETL/ELT Orchestration: Expert proficiency in data transformation engines (such as dbt) and scheduling tools (such as Airflow). Stream/Message Queues: Solid understanding of event‐driven architectures and streaming processing via Apache Kafka. Strong Programming Foundations: Proficiency in Python ...

technical design reviews. Must Haves (Hard Skills) Data Lakehouse Mastery: Proven experience building data infrastructure using Trino/Presto and open table formats like Apache Iceberg or Delta Lake. Advanced Data Modeling: Deep expertise in Medallion Architecture patterns, Star Schemas, and processing semi-structured financial data logs (JSON/… Avro). Robust ETL/ELT Orchestration: Expert proficiency in data transformation engines (such as dbt) and scheduling tools (such as Airflow). Stream/Message Queues: Solid understanding of event-driven architectures and streaming processing via Apache Kafka. Strong Programming Foundations: Proficiency in Python ...

mechanics (order books vs AMMs, fee structures, market manipulation/anti-manipulation signals). Proficiency with SQL; experience with streaming systems (Kafka), workflow orchestration (Airflow), and cloud (AWS/GCP/Azure). Experience with Bayesian methods, probabilistic programming (Stan/PyMC), or ensemble methods. Familiarity with rigorous experimentation … online/offline evaluation, data leakage prevention, and model governance. Tech Stack Python, SQL, pandas/numpy/scipy, PyTorch/sklearn Airflow/dbt, Kafka (or equivalents), Postgres/BigQuery Docker, Kubernetes (optional), CI/CD (GitHub Actions) Observability: Prometheus/Grafana, OpenTelemetry (or equivalents) Locations and Right ...

responsibilities: Building and maintaining data pipelines and services under the guidance of senior engineers. Working with technologies such as Python, SQL, Spark, and Airflow to process large-scale data. Using AI-assisted development tools such as Claude Code, Copilot, or Codex as part of your workflow. Contributing … Basic understanding of data engineering concepts including batch processing and relational databases. Some hands-on experience with data pipeline tools; exposure to Spark or Airflow is a plus. Familiarity with cloud platforms (preferably AWS); production experience is not required. A collaborative mindset and willingness to learn and communicate openly ...

recommendations, ranking and personalization across Bumble products. You will build production-grade models with PyTorch, craft end-to-end ML pipelines using Spark and Airflow, and deploy AI Agents based on Large Language Models, mentoring senior contributors and shaping responsible AI practices. #J-18808-Ljbffr ...

deliver scalable, production-grade Python services and APIs for research and operations. The role emphasizes modern cluster compute technologies (Kubernetes, Spark, Hadoop Yarn, Ray, Airflow) and strong proficiency in Python. A background in mathematics or computing is advantageous. #J-18808-Ljbffr ...

engineers across the firm. Ideal candidates have deep Python expertise, hands-on experience with modern cluster compute technologies (Kubernetes, Spark, Hadoop/Yarn, Ray, Airflow) and a track record of delivering scalable infrastructure in a multi-user #J-18808-Ljbffr ...

topologies. Exposure to MLOps tooling and AI platform layers: experiment tracking (MLflow, W'B), model serving frameworks (Triton, vLLM), and pipeline orchestration (Kubeflow, Airflow). Familiarity with InfiniBand and high-performance networking as it relates to distributed training performance — sufficient to engage credibly with network and infrastructure teams ...

developers, and stakeholders to build APIs and frameworks that scale our investment workflows. You will drive innovative compute solutions, stay current with Kubernetes, Spark, Airflow, and related tools, and help deliver scalable infrastructure across the firm. #J-18808-Ljbffr ...

elevate our current data pipelines and streaming architectures. Establish best practices for monitoring, reliability, and scale across our data ecosystem (using Python, dbt, Airflow, Kafka, and Trino/Iceberg). Build the Unified Customer View: Productionize PoCs into scalable Airflow/dbt data workflows to lay the groundwork … Python and proven experience architecting, deploying, and maintaining Big Data and streaming/batch pipelines (e.g., Kafka Streams, Event Streaming, Trino, Iceberg). dbt & Airflow Proficiency: Hands-on experience using dbt for scalable data transformations and Airflow for workflow orchestration. Data Architecture Mastery: A strong background in designing ...