26 to 50 of 72 Apache Airflow Jobs in London

Owning features end-to-end - from stakeholder requirements through to algorithm hardening, edge-case handling, and value measurement Working with orchestration frameworks (Dagster/Airflow), experiment tracking (MLflow), and containerised infrastructure (Docker/ECS) Collaborating closely with business stakeholders and contributing to roadmap and feature prioritisation What ...

technologies . You will be utilising and playing with tools across the entire data ecosystem. 🚀Essential tech skills include proficiency in Python, Spark, and Airflow , alongside experience with major data platforms such as Snowflake, Databricks, and BigQuery . 🚀Familiarity with data transformation and infrastructure as code tools like ...

modern data warehouse such as Snowflake, Redshift, or BigQuery. Experience building and maintaining ETL pipelines and data models. Knowledge of orchestration tools such as Airflow or Dagster. Infrastructure-as-code experience such as Terraform or similar. A background working in smaller, high-growth companies or strong engineering-led environments. ...

peer to peer feedback and evolving healthy, curious and humble teams. Tech Stack: Python, Javascript/Typescript, React/React Native, AWS, GraphQL, Snowflake, Airflow, DDD. This is an incredible opportunity for a Senior Software Engineer to join a unique company as they embark on a period of significant ...

Testing). Familiarity with C++ or Go is a significant plus. Tooling & Orchestration: Deep experience with containerization (Docker, Kubernetes) and orchestration tools like Kubeflow, Airflow, or Metaflow. Cloud Architecture: Expertise in deploying ML workloads on AWS, GCP, or Azure, utilizing managed services like SageMaker or Vertex AI. Data Foundations ...

Required Data Integration & Ingestion Data Manipulation & Processing GitHub, GitHub Actions, Azure DevOps Azure Data Factory Databricks SQL Database Azure Synapse Stream Analytics AWS Glue Apache Airflow AWS Kinesis Amazon Redshift SonarQube PyTest Optional/Nice to Have Experience managing projects or running Scrum teams Experience with BPC/ ...

Location: London, UK (3 Days/Week Onsite) Type: Fixed Term Contract (6 Months) Mandatory Skills Needed : SQL ( Advanced) , AWS Cloud (Glue,Ethena,EMR), Airflow, Kafka/Kenesis ,Python, Terraform DBT & Snowflake (Basic) AWS Data Engineer Design, develop, and maintain data pipelines for collecting, transforming, and loading data into ...

Greenfield project. What you'll need to succeed Strong Python and SQL are crucial to this role On-prem to Snowflake migration experience Airflow and DBT experience GitHub experience as you will be building CI/CD pipelines as part of this click apply for full job details ...

Greenfield project. What you'll need to succeed Strong Python and SQL are crucial to this role On-prem to Snowflake migration experience Airflow and DBT experience GitHub experience as you will be building CI/CD pipelines as part of this role Any experience within post-trade ...

platforms in a cloud-native environment (AWS, GCP, or Azure). Deep understanding of modern data stacks (e.g. Snowflake/BigQuery/Redshift, dbt, Airflow, BI tools. Comfort operating at both strategic and hands-on levels. Track record of partnering with non-technical stakeholders and influencing at exec level. ...

/lending/risk/portfolio datasets Time series familiarity (trend/seasonality, cohorts over time, monitoring) Experience with data tools such as dbt, Airflow, Snowflake/BigQuery/Databricks (or similar) Experience creating automated data quality frameworks or reconciliation checks Benefits: £45,000 – £55,000 salary Hybrid working ...

mindset, with openness to feedback and diverse perspectives. Bonus points if you have the following. Experience building and maintaining production-grade data pipelines (e.g. Airflow, Dagster) or integrating data from APIs, web scraping, or third-party providers. Familiarity with cloud environments such as Google Cloud or AWS, and deploying ...

data platforms and backend systems. Familiarity with data governance frameworks, privacy compliance, and automated data quality checks. Hands-on experience with big data tools (Airflow, Databricks) and data observability platforms. Collaborative mindset and experience working with cross-functional teams including ML and analytics specialists. Curiosity and enthusiasm for continuous ...

projects are approached and ensuring delivery standards are upheld Designing and governing scalable pipelines using BigQuery, Dataflow, Datastream, Data Fusion, Dataproc and Cloud Composer (Airflow) Leading best practice across dbt, Terraform, Docker and CI/CD Driving Data Mesh adoption, domain ownership, and interoperable data products Setting modelling standards ...

MLflow) Experience with cloud-based ML tools (e.g. SageMaker), data and model versioning (e.g. DVC), CI/CD (e.g. GitHub Actions), workflow orchestration (e.g. Airflow/Dagster) and containerised solutions (e.g. Docker, ECS) nice to have Experience in code testing (unit, integration, end-to-end tests) Strong data engineering ...

Skills & Experience Strong experience with Python and/or SQL in a production environment. Hands-on experience with modern data engineering frameworks such as: Apache Spark, Databricks Airflow/Prefect/Dagster Kafka/EventHub/Kinesis Experience building data solutions on at least one major cloud platform ...

Scala experience Experience delivering and maintaining distributed data processing systems using Big Data tools, lakehouse platforms, and workflow orchestration (experience with Spark, Databricks, or Airflow is a plus) Experience building with cloud platforms such as AWS, Azure, or GCP Experience working with AI tools or platforms (LLMs, predictive models ...

Cloud Run, GKE, CloudSQL, BigQuery) Bare Metal Large datasets & geospatial data (PostGIS a plus) Infrastructure Terraform/Terragrunt GitHub Actions CI/CD Kubernetes Airflow/Prefect/SQLMesh Grafana What We're Looking For 4+ years building production systems Strong React + Python experience Advanced SQL (performance tuning ...

background as a Machine Learning Engineer. Technical Skills: Strong in SQL and Python (Pandas, Scikit-learn, Jupyter, Matplotlib). Data transformation & manipulation : experience with Airflow, DBT and Kubeflow Cloud: Experience with GCP and Vertex AI (developing ML services). Expertise: Solid understanding of computer science fundamentals and time-series ...

technologies including the Python scientific stack (NumPy, Pandas, SciPy, etc.), frameworks such as OpenCV, PyTorch, TensorFlow and JAX, data pipeline tools such as Airflow or Prefect and data processing tools such as Spark or Databricks. The Company The Python Developer will be joining a company whose platform has been ...

directly with clients in a consultative, solution-shaping capacity. * Experience leading or growing technical teams. * Exposure to AWS, automation tools and orchestrators such as Airflow or Dagster. * Understanding of machine learning or predictive analytics, even at a conceptual level. * Experience across multiple industries; familiarity with the music industry ...

candidate has: Experience delivering and maintaining distributed data processing systems using Big Data tools, lakehouse platforms, and workflow orchestration (experience with Spark, Databricks, or Airflow is a plus) Experience building with cloud platforms such as AWS, Azure, or GCP Experience working with AI tools or platforms (LLMs, predictive models ...

keys, sort keys , and workload management. Implement Redshift Spectrum to query data in S3 using external tables. 3. Data Lake & Iceberg Implement and maintain Apache Iceberg tables on AWS for schema evolution and ACID operations. Build Iceberg-based ingestion and transformation pipelines using Glue, EMR, or Spark. Ensure high … monitoring. Required Skills 8 10 years ' experience in data engineering. Strong hands-on experience with: AWS S3, Athena, Glue, Redshift, EMR/Spark Apache Iceberg Python/SQL Experience in telecom data pipelines and handling large-scale structured/semi-structured data. Strong problem-solving, optimization, and debugging skills. ...

services, including indexing, slots, pruning, partitioning, and clustering. Expert-level Kubernetes & GKE resource tuning. Hands-on experience with Dataflow pipelines and worker optimisation. Strong Airflow/Composer knowledge (DAG design, scheduling, PodOperator). Deep understanding of Cloud Logging, routing, sinks, and exclusion filters. Experience with Cloud Spanner autoscaling, indexing ...

high-growth B2B environment Strong technical depth across: Backend: Node js, Typescript Frontend: React, Typescript Database: MySQL Cloud/Infra: AWS, ECS, K8s, Airflow, Kafka, Serverless Solid understanding of SaaS fundamentals: onboarding, authentication, SSO, granular authorisation, multi-tenancy Familiarity with workflow orchestration, asynchronous tasks, and event-driven architecture Ideally ...