76 to 100 of 100 Apache Airflow Jobs in England

culture of engineering excellence, continuous learning, and cross-pollination of knowledge. Own the technical roadmap for Airalo’s data infrastructure (GCP, BigQuery), orchestration (Dagster, Airflow), and ingestion (Fivetran, custom APIs). Drive data-platform architecture decisions, turning ambiguous business problems into scalable, production-grade technical designs. Bridge the data … background with major cloud platforms (GCP preferred) and cloud-native data warehouses (BigQuery preferred, or Snowflake/Redshift). Strong experience with orchestration tools (Airflow, Dagster), ELT pipelines (Fivetran, dbt), and distributed data processing frameworks (Apache Spark, Flink). Hands-on experience using AI tools to accelerate engineering ...

automation, security scanning) Scripting: Bash/Linux/Python Understanding/ability to work in air gapped environments Understanding ML concepts and tools (MLFlow, Airflow) Cloud skills (AWS + Google) Skills: Docker ML Ops AWS Python Bash Kubernetes Docker Containerization Please send CV for full details and immediate interviews. ...

automation, security scanning) Scripting: Bash/Linux/Python Understanding/ability to work in air gapped environments Understanding ML concepts and tools (MLFlow, Airflow) Cloud skills (AWS + Google) Skills: Docker ML Ops AWS Python Bash Kubernetes Docker Containerization If you are interested in this position and would ...

Comfortable acting as a technical authority - advising on approach and best practice and communicating clearly with non-technical stakeholders Experience with Azure, SQL, Dagster, Airflow or similar tools acceptable Experience with Python scripting, React-based applications, FactSet and third-party web APIs Desirable: Understanding of financial data experience with ...

Prepare and manage data across multiple platforms with a focus on accuracy Build and maintain BI dashboards using tools such as QuickSight, DBT, and Airflow Support audit, due diligence, and financial reporting requirements Your Skills and Experience Strong commercial experience in reporting, analytics, and financial analysis Advanced SQL capability ...

influence stakeholders across multiple regions. Technical Environment Experience across some of the following technologies is advantageous: AIops and enterprise automation frameworks Workflow orchestration platforms (Airflow, Argo, Prefect) Configuration and automation tools (Ansible, Salt, Puppet) ITSM and ITOM platforms Event-driven automation and runbook-as-code approaches Messaging technologies such ...

services and tools, such as Salesforce, Segment, HubSpot, Stripe is beneficial Experience with SQL and data warehousing/pipelining tools such as BigQuery, Airflow, etc Proven experience in navigating and transforming legacy codebases, as well as leading greenfield projects ...

Skills (Bonus Points) Experience with MLOps tooling (MLflow, Kubeflow, Weights & Biases, or similar). Building data pipelines (real‐time or batch) using tools like Apache Spark, Kafka, Airflow, or dbt. Familiarity with recommender systems, transformers, or graph neural networks. Exposure to model optimisation techniques (quantisation, distillation, efficient inference ...

Concurrency, Preemption, Stress testing frameworks Testbench Architecture and Implementation GFX API Exposure : Vulkan/DX11/DX12 Exposure Scripting and automation skills (Python, Make, Airflow etc) Embedded FW Development and Debugging Benchmarking and Performance Analysis Windows and/or Linux OS Kernel Architecture C/C++, GNU Toolchain, Visual ...

integration failures, data inconsistencies, and production issues Key Skills • Python & SQL (production-grade pipeline development) • REST APIs, webhooks, OAuth, event-driven architecture • Orchestration tools: Airflow, Prefect, or Dagster • Cloud platforms: AWS, GCP, or Azure • Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills • Vector databases and embedding pipelines • Real-time … robust, production-grade data pipelines from scratch. • Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. • Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). • Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

integration failures, data inconsistencies, and production issues Key Skills Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills Vector databases and embedding pipelines Real-time … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

integration failures, data inconsistencies, and production issues Key Skills Python & SQL (production-grade pipeline development) REST APIs, webhooks, OAuth, event-driven architecture Orchestration tools: Airflow, Prefect, or Dagster Cloud platforms: AWS, GCP, or Azure Docker & Kubernetes Microsoft 365 & Microsoft Copilot Desirable Skills Vector databases and embedding pipelines Real-time … robust, production-grade data pipelines from scratch. Deep familiarity with integration patterns: REST APIs, webhooks, OAuth, and event-driven architectures. Experience with orchestration tools (Airflow, Prefect, or Dagster) and transformation frameworks (dbt or similar). Comfortable across cloud platforms (AWS, GCP, or Azure) and with containerisation (Docker, Kubernetes). ...

My client is based in the London area, is currently looking to recruit an experienced Data Engineer to join their AI Analytics team. They are one of the leaders within the AI space. They are ...

My client is based in the London area, is currently looking to recruit an experienced Data Engineer to join their AI Analytics team. They are one of the leaders within the AI space. They are ...

My client is based in the London area, is currently looking to recruit an experienced Data Engineer to join their AI Analytics team. They are one of the leaders within the AI space. They are ...

focus areas: Building scalable data pipelines using Apache Spark & Airflow Working within a cloud-based, containerised environment (Kubernetes) Contributing to a wider data platform modernisation programme Supporting both batch and streaming workloads Tech: Python, Java, Spark, Airflow, SQL, Cloud (AWS/Azure), Kubernetes (Kafka/streaming nice ...

We're looking for an experienced Analytics Engineer to help shape our data foundations and deliver high quality, trusted insights across the business. You'll design and maintain scalable analytics models, ensure data quality, and ...

code review, source control, build, test, deploy, and operations Awesome If You: Are experienced in Rust/Java/Kotlin Have experience with AWS, Apache Kafka, Kafka Streams, Apache Beam/Flink/Spark – especially deployment, monitoring & debugging Have experience with productisation of Machine Learning research projects … familiar with Airflow or other workflow orchestration tools, and worked with Kubernetes Understand data lake systems and file formats like Parquet, Orc, Athena Have some relevant AWS or Kafka certifications A vibrant, diverse company pushing ourselves and the technology to deliver beyond the cutting edge A team of motivated ...

reliable, production-ready systems that scientists can trust and use daily.You’ll manage and extend a GCP-first platform, working with tools like BigQuery, Airflow, dbt, and Terraform. You’ll also help build internal applications using Python, FastAPI, and React, giving teams easy ways to trigger workflows and access … backend services and working with data workflows A few years of experience with GCP, including BigQuery and cloud-native tools Hands-on experience with Airflow, dbt, and modern data pipelines Infrastructure as code experience, ideally using Terraform Experience building or contributing to internal tools or application Strong ...

Manchester, UK, to develop scalable APIs and manage big data solutions. Candidates should have 8+ years in software or data engineering, with skills in Apache Spark, Airflow, and a strong foundation in Java, Python, or Scala. The role requires collaboration with cross-functional teams to deliver high-quality ...

within air-gapped environments, ensuring secure deployment of ML solutions Collaborate with data scientists to operationalise ML models Utilise tools such as MLflow and Airflow for model tracking and orchestration Support cloud-based deployments across AWS and Google Cloud Platform Ensure performance, reliability, and scalability of machine learning systems … Experience building CI/CD pipelines using GitLab (YAML) Understanding of DevSecOps principles and tooling Exposure to ML lifecycle tools such as MLflow and Airflow Ability to work in highly secure or air-gapped environments Strong problem-solving and collaboration skills What you'll get in return Opportunity ...

distributed processing frameworks such as Spark or Databricks Build and optimise data workflows across cloud platforms (AWS, Azure, GCP) Implement orchestration frameworks such as Airflow Test, troubleshoot, and improve data products to ensure reliability and quality Collaborate with engineering teams to deliver robust, maintainable systems Key Requirements: Experience building … Experience working with at least some of: Distributed computing frameworks (e.g. Spark, Databricks) Cloud platforms (AWS, Azure, or GCP) Workflow orchestration tools (e.g. Airflow) Strong understanding of data system reliability, testing, and performance Experience troubleshooting and improving production data systems Strong problem-solving skills and attention to detail Ability ...

distributed processing frameworks such as Spark or Databricks Build and optimise data workflows across cloud platforms (AWS, Azure, GCP) Implement orchestration frameworks such as Airflow Test, troubleshoot, and improve data products to ensure reliability and quality Collaborate with engineering teams to deliver robust, maintainable systems Key Requirements: Experience building … Experience working with at least some of: Distributed computing frameworks (e.g. Spark, Databricks) Cloud platforms (AWS, Azure, or GCP) Workflow orchestration tools (e.g. Airflow) Strong understanding of data system reliability, testing, and performance Experience troubleshooting and improving production data systems Strong problem-solving skills and attention to detail Ability ...

ingested data to business-ready datasets Optimise PySpark-based data transformation logic for performance and reliability Build scalable and maintainable pipelines in Databricks and Airflow Implement and uphold GDPR-compliant processes around PII data Collaborate with stakeholders to define what "business-ready" means, and confidently sign off datasets … Experience Required Extensive hands-on experience with PySpark, including performance optimisation Deep working knowledge of Databricks (development, architecture, and operations) Proven experience working with Airflow for orchestration Proven track record in managing and securing PII data, with GDPR compliance in mind Experience in data governance processes; Alation experience preferred ...

DRW Holdings, LLC. is seeking a Senior Software Engineer for their Unified Platform in London. The role involves building robust systems with an emphasis on testing and scalability, designing and programming in languages such as ...