51 to 75 of 141 Apache Airflow Jobs in London

organisation* Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI* Expertise in Streaming Data, Event Driven Design, Data Governance ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

across the data engineering domain. Engage with data engineering technical detail when needed, understanding the data platform landscape including data ingestion, transformation (DBT), orchestration (Airflow), streaming (Kafka), and data warehousing (Snowflake), and able to articulate technical choices and trade-offs to different stakeholders. Work at both conceptual and detailed … despite complexity in data requirements and priorities. What I need to know: Strong understanding of data engineering technologies including data ingestion, transformation (DBT), orchestration (Airflow/Astronomer). Experience with cloud data platforms (AWS data services, Azure data services) including cost management and optimization Knowledge of data quality frameworks ...

Strong hands‐on background with modern cloud data platforms (AWS/GCP/Azure) Expertise across Python, SQL and data orchestration tools such as Airflow Experience building scalable data pipelines and data platforms in a fast‐paced fintech or technology environment A collaborative leadership style with the ability ...

Strong hands-on background with modern cloud data platforms (AWS/GCP/Azure) Expertise across Python, SQL and data orchestration tools such as Airflow Experience building scalable data pipelines and data platforms in a fast-paced fintech or technology environment A collaborative leadership style with the ability ...

implementing large-scale data pipelines, ETL processes, and distributed systems Should be able to work independenty with minimal help/guidance Good Understanding of Airflow,Data Fusion and Data Flow Strong Background and experience in Data Ingestions,Transformation,Modeling and Performance tuning. One migration Experience from Cornerstone ...

Experience Data Modeling Data Integration & Ingestion Data Manipulation and Processing GitHub, GitHub Actions, Azure DevOps Data Factory, Databricks, SQL DB, Synapse, Stream Analytics, Glue, Airflow, Kinesis, Redshift, SonarQube, PyTest AI based Code developed and AI-built automated data pieplines Key Characteristics Technology expert who constantly pursues knowledge enhancement ...

improvements What will set you apart Hands-on experience with the modern data stack, for example Iceberg or Delta, Snowflake or Databricks, dbt or Airflow Experience with a data migration or re-platforming project, on either the product or the engineering side Experience in financial services or another regulated ...

management. Data Center Optimization Collaborate with facilities and operations teams to optimize infrastructure deployments based on available rack space, power draw, thermal dynamics, and airflow efficiency. Contribute to the planning and provisioning of data center resources, ensuring support for both traditional workloads and AI‐intensive environments. Evaluate emerging technologies ...

visualisation languages and libraries (SQL, python, matplotlib, etc.) as well as some of the supporting tools and platforms (e.g. git, MLFlow, Kubernetes, Kubeflow, Trino, AirFlow). A strong understanding of statistical concepts (hypothesis testing, sampling, probability distributions). Excellent communication and collaboration skills to effectively bridge the gap between ...

unstructured data sources including Cassandra HBase Kafka and Spark* Strong ETL and data pipeline experience particularly with modern big data toolsets such as Airflow and Nifi or equivalent* Strong programming background in either Java Python or Scala* Experience in modern engineering practices and technologies* Good working knowledge of data ...

Cloud Run), AWS (Bedrock, SageMaker) Experience building and automating AI/ML pipelines using tools such as MLflow, Kubeflow, Azure ML, Vertex Pipelines, Airflow or Google ADK Hands-on experience with Generative and Agentic AI frameworks such as LangChain, LlamaIndex, CrewAI, Autogen, Google ADK, or similar. Ability to design ...

bring mission‐critical software, analytics, and AI to life and is the company behind Astro, the industry‐leading unified DataOps platform powered by Apache Airflow®. Astro accelerates building reliable data products that unlock insights, unleash AI value, and powers data‐driven applications. Trusted by more than … building and scaling AE and Sales Engineering functions in a high‐growth environment. Preferred Qualifications Experience selling into the modern data stack ecosystem (Airflow, dbt, Snowflake, Databricks, Kafka, or similar). Familiarity with open‐source‐led or product‐led growth (PLG) sales motions — understanding how to convert community users ...

will shape what modern data infrastructure looks like in this space. You’ll take full ownership of the backend, building custom data pipelines, orchestrating Airflow‐style workflows, and turning messy real‐world inputs into structured data ready for downstream AI models. You’ll design APIs, optimise low‐latency queries ...

fast-moving fintech environment What they’re looking for: Strong SQL and some Python experience Exposure to modern data tooling such as dbt or Airflow Some experience with data visualisation and reporting Strong communication skills, especially with non-technical stakeholders High EQ, curiosity, and willingness to learn A collaborative ...

cloud‐native architectures, with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats ...

abstraction layers Model the cost, feasibility and risk of different LLMs and implementation approaches THE STACK AWS, Bedrock, LangGraph, LangChain, Python, FastAPI, PostgreSQL, Snowflake, Airflow, React, Next.js, Kubernetes. YOUR EXPERIENCE 15+ years in enterprise software engineering with a background in product engineering organisations or technology companies Deeply technical: recent ...

willingness to take initiative. Desirable Experience with dashboarding/BI tools (Power BI, Tableau, Streamlit). Familiarity with Linux scripting or workflow orchestration tools (Airflow, Prefect). Exposure to financial markets , asset management, or consulting environments. This role would suit someone who is technically strong, logical, and ambitious, with ...

cloud-native architectures, with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats ...

cloud-native architectures, with experience shaping technical direction across teams and stakeholders Solid expertise in relevant technologies such as AWS, Terraform, Kafka/MSK, Airflow, Glue or Spark, and building scalable data solutions Experience with modern data ecosystems including data lakes/lakehouse architectures, Iceberg or similar table formats ...

similar databases. Experience designing data models, schemas and transformation layers for complex datasets. Experience with directed acyclic graph-based orchestration tools such as Airflow, Dagster, Prefect or similar systems. Familiarity with data warehouses, object storage, queues, APIs and cloud infrastructure. Experience with data quality testing, lineage, observability, versioning ...

Ability to diagnose faults and remedy under time pressure Additional Beneficial Skills or Experience Experience with stream processing (Kafka, Flink) Experience with dbt, Snowflake, Airflow Experience with other programming languages Python, Java, Scala, F# Kubernetes/OpenShift/Docker, helm templates Perks & Benefits Annual Discretionary Bonuses Annual Training Allowance ...

forecasting, and statistical modeling in a product environment. Expertise in Python or R, fluency in SQL, and experience with large‐scale data tech (Databricks, Airflow, dBT a plus). Experience with FinTech, payments, crypto, or blockchain data is a strong plus. Advanced degree (MS, PhD) in a quantitative field ...

tools like Looker, Grafana, Lightdash, or Superset Ability to build and manage data pipelines that are modularised and scalable, using tools like DBT and Airflow Familiarity with observability/reliability concepts (SLIs, SLOs, incidents) Some experience with Python/data transformation (DBT, etc.) is helpful This ...

alongside a clear sense of when simpler rules-based approaches are the better choice Experience with cloud data tooling such as BigQuery, Spark, dbt, Airflow or similar Experience building tooling for operational or investigative teams, and partnering closely with the people who use it Interest in AI safety ...