26 to 50 of 51 Apache Airflow Jobs in London

Docker/Kubernetes/Helm/Flux Data visualisation tools and notebooking tools such as Marimo/Jupyter ETL/ELT technologies such as Airflow/Dagster/DBT Offered World-class tech environment, culture and infrastructure, working on cutting-edge projects Work on solutions with clear, direct business ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

Cloud Run), AWS (Bedrock, SageMaker) Experience building and automating AI/ML pipelines using tools such as MLflow, Kubeflow, Azure ML, Vertex Pipelines, Airflow or Google ADK Hands-on experience with Generative and Agentic AI frameworks such as LangChain, LlamaIndex, CrewAI, Autogen, Google ADK, or similar. Ability to design ...

retrieval augmented generation (RAG) architectures. * Strong knowledge of cloud platforms including AWS or Azure. * Experience with data engineering and orchestration tools such as Databricks, Airflow, Docker, and Kubernetes is desirable. * Must have previous experience working within a small to medium-sized business (SME), start-up, or scale-up environment. ...

platforms and pipelines (batch and real-time ingestion). Hands-on familiarity with: Snowflake (or equivalent cloud data warehouse), Looker (or other BI tools), Airflow, Pub/Sub, or similar orchestration and ingestion frameworks. Experience with data infrastructure monitoring and optimization. Strong communication skills. Ability to translate business goals ...

complex datasets. You can write optimized SQL queries to extract "financial truth". Exposure to other technical skills considered a plus (Dbt, Python, Airflow, BI tools). Foundational Finance Literacy: You don’t need to have a finance/accounting background, but you are comfortable with basic financial concepts. ...

tools like Looker, Grafana, Lightdash, or Superset Ability to build and manage data pipelines that are modularised and scalable, using tools like DBT and Airflow Familiarity with observability/reliability concepts (SLIs, SLOs, incidents) Some experience with Python/data transformation (DBT, etc.) is helpful This ...

GitHub Copilot) with significant examples of delivery improvement AWS (S3, Lambda, EC2, Glue), Snowflake, Git, CI/CD, Datadog, MS SQL Server, REST, GraphQL, Airflow (as many as possible) Degree educated or higher in a relevant technical discipline from a leading university This is a £650-700/ ...

creating. You'll have experience with: AWS and cloud-native technologies Snowflake or similar cloud data platforms Data orchestration tools such as Airflow, Prefect or similar Amazon S3 and modern data architectures Postgres and semi-structured data models Event-driven architectures and scalable systems You'll understand how data ...

happens next. Technical Mastery : You bring expert-level SQL skills and are fully comfortable building and maintaining your own robust data pipelines using dbt, Airflow, and Snowflake. Beyond querying, you use Python for advanced automation and forecasting, and you apply statistical rigor (e.g., Bayesian statistics, causal inference) to solve ...

progress; evaluating the impact of what you shipped; recommending changes and new initiatives. Analytical thinker with strong analysis skills (SQL, Python), data modelling (e.g. airflow, dbt) and experience using event data (e.g. Mixpanel). Hands-on experience designing and running experiments. You know how to design an experiment ...

working on the most impactful opportunities NICE TO HAVE BUT NOT ESSENTIAL Experience delivering high quality data and self-serve analytics using tools like Airflow, dbt and Looker. Practically used + applied AI/machine learning techniques and built predictive models to get deeper insights Knowledge of the FinTech ...

mentality. You can take work beyond the analysis and get things done. Good to have: Experience in the Fintech domain Knowledge of utilities like Airflow/Github/DBT Experience with user-facing products and data Qualifications Additional Information Some of our benefits: 💰Base salary ...

frequently, and drive actions across teams. Good to have FinTech/payments and Fraud experience. Experience with feature engineering and model monitoring. Familiarity with Airflow/DBT/Git/Snowflake and production data platforms. If you care about protecting customers and enabling growth through rigorous, product-focused fraud ...

Robert Half Technology are assisting a rapidly growing sports fashion brand to recruit a Data Engineer on a contract basis. This is an exciting opportunity to join a fast-paced SME environment at a pivotal ...

they build. What You’ll Do Build and own data pipelines that connect product, analytics, and operations Design scalable architectures using tools like dbt , Airflow , and Snowflake/BigQuery Work with engineers and product teams to make data easily accessible and actionable Help evolve their data warehouse and ensure … decisions across the stack — they love new ideas What You Bring Hands-on experience in Python and SQL Experience with modern data tools (dbt, Airflow, Prefect, Dagster, etc.) Knowledge of cloud platforms like AWS , GCP , or Azure An understanding of data modelling and ETL best practices Curiosity, creativity ...

data flow, including supporting delivery in air-gapped and other restricted environments. Delivering data applications in an automated and continuous fashion. Utilising tools like Airflow for orchestration and automating data pipeline deployments using DevOps tools and techniques (Docker, Containers, Terraform, Kubernetes) Building data pipelines using both cloud-native … experience as a DataOps Engineer delivering DataOps solutions in public sector or similarly regulated environments. Developing, orchestrating and maintaining data pipelines using tools like Airflow, Prefect, or Dagster. Containerising data applications using Docker and deploying them to Container Platforms (EKS, AKS and Kubernetes). Implementing CI/CD pipelines ...

Prepare and manage data across multiple platforms with a focus on accuracy Build and maintain BI dashboards using tools such as QuickSight, DBT, and Airflow Support audit, due diligence, and financial reporting requirements Your Skills and Experience Strong commercial experience in reporting, analytics, and financial analysis Advanced SQL capability ...

Develop batch and ELT/ETL data workflows for ingesting and processing large datasets Work with technologies such as Python, Java, Spark, SQL, and Airflow Apply good software engineering practices, including testing, documentation, and CI/CD Contribute to cloud‐based systems (AWS), focusing on reliability, monitoring, and automation … data engineering concepts such as data lakes, batch processing, and distributed systems Experience building or operating data pipelines using tools like Spark and Airflow Familiarity with cloud platforms (AWS preferred) and production monitoring Clear communication skills and a collaborative approach to problem‐solving Why join us? Join our team ...

integration failures, data inconsistencies, and production issues Key Skills • Python & SQL (production-grade pipeline development) • REST APIs, webhooks, OAuth, event-driven architecture • Orchestration tools: Airflow, Prefect, or Dagster • Cloud platforms: AWS, GCP, or Azure • Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills • Vector databases and embedding pipelines • Real-time … robust, production-grade data pipelines from scratch. • Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. • Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). • Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

integration failures, data inconsistencies, and production issues Key Skills Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills Vector databases and embedding pipelines Real-time … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

high-impact, early-stage business £100K plus equity in a well-funded Series A start-up. Own and shape a GCP data platform (BigQuery, Airflow, dbt). Real progression into a future leadership role. We’re working with an early-stage, well-funded tech-led start-up. They … production-ready systems that scientists can trust and use daily. You’ll manage and extend a GCP-first platform, working with tools like BigQuery, Airflow, dbt, and and so much more. Sound like you? What experience you’ll need to apply Strong Data Engineering experience in a cloud environment ...

reliable, production-ready systems that scientists can trust and use daily.You’ll manage and extend a GCP-first platform, working with tools like BigQuery, Airflow, dbt, and Terraform. You’ll also help build internal applications using Python, FastAPI, and React, giving teams easy ways to trigger workflows and access … backend services and working with data workflows A few years of experience with GCP, including BigQuery and cloud-native tools Hands-on experience with Airflow, dbt, and modern data pipelines Infrastructure as code experience, ideally using Terraform Experience building or contributing to internal tools or application Strong ...

Senior Data Engineer | Python | Airbyte | Apache | BigQuery | EducationTech | Remote, London | £70k Looking to help shape Data Strategy for a scaling Tech-for-Good business I'm partnered with a AI-first data and analytics platform built for schools, multi-academy trusts and education groups. Their technology connects fragmented school … Engineer to build and maintain the data infrastructure that underpins the entire platform.You’ll own and extend the ELT pipeline - Airbyte (OSS) for extraction, Airflow on Astro for orchestration, dbt via Cosmos for transformation - and the BigQuery lakehouse beneath it, with a semantic layer on top.These are hands ...

This job is with Wise, an inclusive employer and a member of myGwork – the largest global platform for the LGBTQ+ business community. Please do not contact the recruiter directly. Company Description Wise is a global ...