1 to 25 of 56 Apache Airflow Jobs in London

data platform and engineering capability as the business continues to scale its trading operations. They are looking for a Data Engineer with strong Apache Airflow experience to help design and build the pipelines that power trading, analytics and commercial decision-making. This is a great opportunity to join … platform and the opportunity to work with modern open-source technologies . What You’ll Be Doing Designing and building data pipelines orchestrated with Apache Airflow Developing ETL/ELT workflows in Python and SQL to process trading and operational data Building reliable, scalable data infrastructure to support ...

Platform DevOps team to build and run highly reliable CI/CD pipelines and infrastructure for the bank’s modern data estate (AWS + Airflow + Databricks). Key Responsibilities Design, build and maintain Infrastructure as Code using CloudFormation and AWS services Manage and scale Apache Airflow … call rotation and incident resolution Essential Skills & Experience 5+ years as a DevOps/Platform Engineer Strong AWS experience + CloudFormation Production experience with Apache Airflow (MWAA) Lambda , Linux + Bash scripting CI/CD expertise (Git, Jenkins, GitHub Actions) IaC and GitOps practices Nice to Have Terraform ...

Platform DevOps team to build and run highly reliable CI/CD pipelines and infrastructure for the bank’s modern data estate (AWS + Airflow + Databricks). Key Responsibilities Design, build and maintain Infrastructure as Code using CloudFormation and AWS services Manage and scale Apache Airflow … call rotation and incident resolution Essential Skills & Experience 5+ years as a DevOps/Platform Engineer Strong AWS experience + CloudFormation Production experience with Apache Airflow (MWAA) Lambda , Linux + Bash scripting CI/CD expertise (Git, Jenkins, GitHub Actions) IaC and GitOps practices Nice to Have Terraform ...

governance, security, and regulatory compliance . Ability to work closely with technical and business teams to deliver impactful solutions. Nice-to-Haves: Orchestration tools (Airflow, dbt, Prefect) or streaming frameworks (Kafka, Kinesis). CI/CD experience and DevOps practices for data workflows. Exposure to data visualisation platforms (PowerBI ...

Location: London, UK (3 Days/Week Onsite) Type: Fixed Term Contract (6 Months) Mandatory Skills Needed : SQL ( Advanced) , AWS Cloud (Glue,Ethena,EMR), Airflow, Kafka/Kenesis ,Python, Terraform DBT & Snowflake (Basic) AWS Data Engineer Design, develop, and maintain data pipelines for collecting, transforming, and loading data into ...

architecture to support data consumption for reporting systems, operational applications, and analytics platforms. Implement automation, orchestration, and scalable pipeline frameworks using tools such as Apache Airflow. Collaborate with business stakeholders and technical teams to align the data platform with enterprise data strategy. Support hospitality analytics use cases, including guest … predictive analytics, recommendation engines, and customer behaviour analytics. Ability to design AI-ready data platforms for future analytics use cases. Preferred Skills Experience with Apache Airflow for pipeline orchestration. Knowledge of CI/CD pipelines, DevOps, and Git-based development workflows. Experience with data governance, metadata management ...

data models and map various data sources to support downstream analytics. Pipeline Engineering: Build scalable ETL pipelines and data quality solutions using tools like Airflow, Luigi, or Azkaban. Codification: Translate intricate business logic into high-quality code using Python, Scala, or Java. Innovation: Evaluate and implement new technologies across ...

models are always running, always accurate, and always improving. Key Responsibilities ML Pipeline Orchestration & Automation Own and extend Circadia’s ML pipeline orchestration using Apache Airflow, including training, evaluation, and deployment workflows. Build and maintain automated pipelines for model retraining, validation, and promotion across development, staging, and production … closely related infrastructure role. Strong proficiency in Python for ML pipeline development, tooling, and automation. Hands-on experience with ML pipeline orchestration tools, particularly Apache Airflow. Experience with model registries and experiment tracking platforms (MLflow preferred). Experience deploying and operating ML workloads on AWS (Batch ...

Skills and Experience * Strong commercial experience working with AWS in production environments. * Proficiency in Python and SQL for data processing and orchestration. * Experience with Airflow for workflow management. * Knowledge of cloud data warehousing, ideally Redshift. * Experience with infrastructure as code and CI/CD tooling such as Terraform ...

with dimensional modelling, star schemas, SCDs etc.Experience within regulated financial servicesDesirableExperience in pensions, asset management or institutional investingUnderstanding of performance measurement and attributionExposure to Airflow or dbtCloud platform familiarity (Azure or AWS)Knowledge of data governance best practiceAbout YouHighly detail-driven with a strong approach to data qualityComfortable operating ...

1classification ) Strong SQL&Python Hands-onETL/ELTexperience(e.g.,Matillion,Talend,Fivetran,ADF) Experiencewith Snowflake (AWS/Azure/GCPbeneficial) Strongdatamodellingknowledge Understandingofdatagovernanceandsecuritybestpractices Abilitytotranslatebusinessneedsintoscalablesolutions Desirable Airflow,dbt,Kafka,CI/CD,BItools,governanceplatforms,financialservicesexperience. Interestedinbuildingmodern,enterprise-scaledataplatformsinahighlyregulatedenvironment Applynow ...

experience Time-series experience (market data, telemetry, pricing, events) Streaming exposure (Kafka/Event Hubs/Kinesis) Experience with Temporal (or similar orchestrators like Airflow/Dagster/Prefect) Any exposure to AI agents/automation tooling 👉 Apply ...

with SQL and modern data warehousing technologies (e.g., Snowflake, BigQuery, Redshift, Databricks). Proven experience building production-ready pipelines using tools such as dbt, Airflow, Dagster, Prefect, or similar. Experience integrating data from HRIS and talent systems (e.g., Workday, BambooHR, Greenhouse, Lever, HiBob, Personio). Ability to work with ...

cloud data platforms (AWS preferred) Deep experience with AWS, Databricks, Power Platform, and Redshift (Snowflake a plus) Proficiency in AWS Glue, Qlik Talend, DBT, Airflow, and modern data integration tools. Excellent knowledge of Python, SQL, PowerQuery (M), and preferably Scala or PySpark Working knowledge with enterprise architecture frameworks (e.g. ...

Proven experience in a Lead/Senior Data Engineering role Strong handson experience with Snowflake Expertise in ETL/ELT and orchestration tools (e.g. Airflow, dbt, Matillion) Strong SQL and Python skills Experience building scalable data platforms in cloud environments Previous experience leading or mentoring engineers Strong communication ...

Owning features end-to-end - from stakeholder requirements through to algorithm hardening, edge-case handling, and value measurement Working with orchestration frameworks (Dagster/Airflow), experiment tracking (MLflow), and containerised infrastructure (Docker/ECS) Collaborating closely with business stakeholders and contributing to roadmap and feature prioritisation What ...

performance marketing data Experience working on large-scale data transformation or platform modernisation projects Tech Stack Warehouse: Snowflake Transformation: dbt BI: Looker Orchestration: Airflow Version Control: GitHub Other Tools: Adverity, OpenMetadata, Jira, Confluence Benefits Hybrid working (minimum of 3 days per week in office) 25 days annual leave plus ...

Overview Lead a team of Data Analysts and own the delivery of high-impact analytical products using modern data engineering tooling (AWS, Databricks, dbt, Airflow). You will bridge the gap between business stakeholders and the data platform. Key Responsibilities Lead and mentor a team of Data Analysts Build … maintain self-serve analytical assets using dbt models and Databricks Design and orchestrate analytical pipelines in Airflow Translate complex business requirements into actionable insights and dashboards Perform deep-dive analysis across Retail, Commercial, Risk and Marketing domains Ensure all deliverables meet strict data governance and regulatory standards Work closely ...

Tech Stack Language: Expert-level Python and advanced SQL . Cloud: Extensive experience with GCP (BigQuery, Cloud Storage, Dataflow/PubSub, Cloud Composer/Airflow). Tools: Experience with data orchestration tools and version control (Git). Environment: Proficiency in building production-grade pipelines. What We’re Looking ...

cross-functional teams. Strong stakeholder communication skills at all levels. Comfortable leading iterative delivery using Agile practices. Exposure to tools such as dbt, Airflow/ADF/Dagster, Snowflake/Databricks/Fabric, and metadata/governance systems is beneficial. Ready to Lead in a Role with Real Influence ...

mindset, with openness to feedback and diverse perspectives. Bonus points if you have the following. Experience building and maintaining production-grade data pipelines (e.g. Airflow, Dagster) or integrating data from APIs, web scraping, or third-party providers. Familiarity with cloud environments such as Google Cloud or AWS, and deploying ...

Location: London, UK (3 Days/Week Onsite) Type: Fixed Term Contract (6 Months) Mandatory Skills Needed : SQL ( Advanced) , AWS Cloud (Glue,Ethena,EMR), Airflow, Kafka/Kenesis ,Python, Terraform DBT & Snowflake (Basic) AWS Data Engineer Design, develop, and maintain data pipelines for collecting, transforming, and loading data into ...

Greenfield project. What you'll need to succeed Strong Python and SQL are crucial to this role On-prem to Snowflake migration experience Airflow and DBT experience GitHub experience as you will be building CI/CD pipelines as part of this role Any experience within post-trade ...

MLflow) Experience with cloud-based ML tools (e.g. SageMaker), data and model versioning (e.g. DVC), CI/CD (e.g. GitHub Actions), workflow orchestration (e.g. Airflow/Dagster) and containerised solutions (e.g. Docker, ECS) nice to have Experience in code testing (unit, integration, end-to-end tests) Strong data engineering ...

team of 6–10 Data Engineers (onshore + offshore) Own the technical roadmap for the data platform (AWS + Databricks + dbt + Airflow) Architect scalable, cost-efficient and governed data solutions Drive best practices in engineering, testing, observability and documentation Act as senior technical point of contact … Experience 8+ years total experience with at least 3 years in a Lead/Principal Data Engineer role Expert-level AWS , Databricks , dbt and Apache Airflow Proven experience leading teams in a regulated financial services environment Strong stakeholder management and ability to translate business needs into technical solutions ...