51 to 72 of 72 Apache Airflow Jobs in London

high-growth B2B environment Strong technical depth across: Backend: Node js, Typescript Frontend: React, Typescript Database: MySQL Cloud/Infra: AWS, ECS, K8s, Airflow, Kafka, Serverless Solid understanding of SaaS fundamentals: onboarding, authentication, SSO, granular authorisation, multi-tenancy Familiarity with workflow orchestration, asynchronous tasks, and event-driven architecture Ideally ...

clearly. Excellent problem-solving skills and the ability to translate business requirements into technical solutions. Languages: Python (primary), SQL, Bash Cloud: Azure, AWS Tools: Airflow, DBT Data Platforms: Snowflake, Delta Lake, Redis, Azure Data Lake Infrastructure and Operations: Terraform, GitHub Actions, Azure DevOps, Azure Monitor Desirable Skills and Experience … platforms such as Snowflake and Azure Data Lake. Understanding of monitoring, model performance tracking and observability best practices. Familiarity with orchestration tools such as Airflow or Azure Data Factory. ...

Platforms Itential Automation Platform Cisco NSO – Service development and lifecycle Apache Airflow Hasi Corp Vault Openshift Automation development Python Fast API Rest API Netconf Yang modelling XML/Json Jinja/TextFSM Pytest Unit testing Database MongoDB DevOps Docker Git CI/CD Kubernetes Networking Ciena 517x/ ...

/Gas), who is technically well versed in systematic trading development, and analysis data-sets (time series, market-data). Stack: Python, AWS, ETL, Airflow The team has a flat structure, where developers are interfaced with PMs and traders, to ensure the software systems created are bespoke. ...

Senior Data Engineer (AWS, Airflow, Python) Based at client locations, working remotely, or based in our Godalming or Milton Keynes offices. Salary up to £65k plus company benefits. About Us Triad Group Plc is an award-winning digital, data, and solutions consultancy with over 35 years' experience primarily serving … maintain cloud data solutions. Maintain modular Terraform based IaC for reliable provisioning of AWS infrastructure. Develop, optimise and maintain robust data pipelines using Apache Airflow. Implement data transformation processes using Python to clean, preprocess, and enrich data for analytical use. Collaborate with data analysts, data scientists, developers, and other ...

including managing dependencies and build pipelines without internet access Proven background building APIs for data enrichment services Orchestration & data pipeline experience with Prefect or Airflow, and K8s or Swarm Configuration and automation of business workflow tooling (e.g., Jira) Hands-on experience with a range of databases and storage technologies ...

Linux environment. - Experience in building automation tools and managing system configurations. - Knowledge of C++, KDB/q, and experience with technologies like Slurm, Airflow, Kafka, or AMPS. - Background in enhancing system stability, scalability, and performance while conducting root cause analyses to resolve incidents efficiently. - Observability skillset, monitoring and analysis ...

drive business impact through data Technical Environment Languages: Python, SQL, Bash Cloud: Azure, AWS Data & Platforms: Snowflake, Delta Lake, Redis, Azure Data Lake Tools: Airflow, DBT, Docker Infrastructure & Operations: Terraform, GitHub Actions, Azure DevOps, Azure Monitor Desirable Skills Experience with data platforms such as Snowflake or Azure Data Lake … Experience deploying models as APIs (e.g. FastAPI, Azure Functions) Knowledge of monitoring, model performance tracking and observability practices Familiarity with orchestration tools such as Airflow or Azure Data Factory ...

What you'll be doing: Design, develop, and maintain scalable data architectures and ETL pipelines Build and manage data models and data warehouse solutions (Airflow, dbt, and Redshift) Write clean, efficient Python and SQL code for data processing and transformation Integrate data from internal and third-party APIs … principles Ability to optimise data pipelines for performance and maintainability Strong collaboration skills and a proactive, problem-solving mindset Bonus Points For Experience with Airflow and/or dbt Experience working in Agile environments (Scrum/Kanban) Exposure to DevOps practices or CI/CD pipelines You'll join ...

ingestion workflows Implement data quality frameworks and enforce data contracts Build CI/CD pipelines and automated testing for data workflows Orchestrate pipelines using Airflow, Prefect, or Dagster Improve observability, monitoring, and lineage across data systems Mentor junior engineers and elevate engineering standards Key Skills and Experience Management … Data Engineering experience Expert-level SQL and advanced data modelling Strong Python Experience with Snowflake and Databricks Cloud experience Experience with orchestration tools (Airflow, Prefect) CI/CD and DataOps experience for data platforms This role will suit someone who is looking to remain hands on , whilst leading ...

ingestion workflows Implement data quality frameworks and enforce data contracts Build CI/CD pipelines and automated testing for data workflows Orchestrate pipelines using Airflow, Prefect, or Dagster Improve observability, monitoring, and lineage across data systems Mentor junior engineers and elevate engineering standards Key Skills and Experience Management … Data Engineering experience Expert-level SQL and advanced data modelling Strong Python Experience with Snowflake and Databricks Cloud experience Experience with orchestration tools (Airflow, Prefect) CI/CD and DataOps experience for data platforms This role will suit someone who is looking to remain hands on , whilst leading ...

skills with the ability to work cross-functionally with stakeholders, engineers, and analysts. Technical Stack: Languages: Python (primary), SQL, Bash Cloud: Azure, AWS Tools: Airflow, DBT, Docker Data: Snowflake, Delta Lake, Redis, Azure Data Lake Infra & Ops: Terraform, GitHub Actions, Azure DevOps, Azure Monitor Desirable skills/knowledge/… Experience deploying models as APIs (FastAPI, Azure Functions). Understanding of monitoring, model performance tracking, and observability best practices. Familiarity with orchestration tools like Airflow or Azure Data Factory. ...

Group) I'm hiring a Mid-Level Data Engineer (3+ years’ experience) to help scale and optimise a robust data platform built on Python, Airflow, dbt and Redshift. 🔧 What you’ll be doing: • Scaling and optimising ELT pipelines (Python + Airflow) • Enhancing the data warehouse (performance, cost, complexity … looking for: • 3+ years in Data Engineering (FinTech/startup/scale-up ideal) • Strong Python + SQL expertise • Production-grade workflow orchestration (Airflow/Dagster/Prefect) • Deep understanding of data warehousing + ELT principles • Interest in Open Banking & payments This is a pivotal role at a high ...

delivering advanced data solutions that support the future data strategy of the organisation. You will harness modern technologies including Data Vault, Snowflake, DBT, Airflow and AWS/Azure to shape scalable, future-proof data architectures. In this role, you will be responsible for defining, owning and governing data models … translate business requirements into effective data architecture solutions. Experience working with or supporting cloud migrations. Languages: Python (primary), SQL, Bash Cloud: Azure, AWS Tools: Airflow, DBT, Docker, Fargate, FastAPI Data Platforms: Snowflake, Delta Lake, Redis, Azure Data Lake Infrastructure and Operations: Terraform, GitHub Actions, Azure DevOps, Grafana, Azure Monitor ...

Gateway, and Kinesis. Integrating third-party APIs into the data platform and transforming data for CRM delivery. Migrating R-based data streams into modern Airflow-managed Python/DBT pipelines. Ensuring observability and reliability using CloudWatch and automated monitoring. Supporting both BAU and new feature development within the data … services including Lambda, API Gateway, S3, Kinesis, and CloudWatch. Strong programming ability in Python and data transformation skills using SQL and DBT. Experience with Airflow for orchestration and scheduling. Familiarity with third-party API integration and scalable data delivery methods. Excellent communication and the ability to work ...

training, validation, deployment, and monitoring of models. Support scalable ML solutions across use cases like recommendations, forecasting, and automation. Work with tools like Airflow, Azure ML, and FastAPI to deliver production-ready systems. Automate model deployment workflows using CI/CD tools (e.g. GitHub Actions, Azure DevOps). Contribute … code (Terraform). Collaborative team player with excellent communication skills. Technical Stack: Languages: Python (primary), SQL, Bash Cloud: Azure, AWS Tools: MLflow, Azure ML, Airflow, Docker, Fargate, FastAPI Data: Snowflake, Delta Lake, Redis, Azure Data Lake Infra & Ops: Terraform, GitHub Actions, Azure DevOps, Grafana, Azure Monitor Desirable skills/ ...

Senior Data Engineer AI SaaS | Data as a Product | London (Hybrid) This is a brilliant opportunity for someone who loves solving problems at the intersection of data and product and wants real ownership in a ...

Technical Lead/Data Architect Start date: ASAP Duration: 6 months Location: London, flexible working *Please note. - You must be registered as a freelance contractor (Ltd Co, Sole Trader) or work via an umbrella company ...

Uses Python day-to-day Has operated BigQuery at scale , and understands how queries behave across hundreds of terabytes Has built production systems with Airflow/Composer , DBT , Terraform and Docker Is comfortable with GKE , Cloud Run or Cloud Functions Has worked with streaming systems built on Dataflow/… Apache Beam Can work closely with non-technical stakeholders and shape solutions rather than just execute tickets Why join? Clear progression path - report directly into CTO with a clear route to a leadership position An opportunity to take ownership of the data platform Be part of a highly motivated ...

than headcount substitution, aligning with Outside IR35 status . Key responsibilities include: Designing, building, and delivering data pipelines and orchestration workflows using Prefect or Airflow Developing Python-based services and APIs for data ingestion and enrichment Building and deploying containerised workloads using Docker and Kubernetes (or Swarm) Managing dependencies … UKSV DV clearance (minimum 12 months remaining) Strong Python development experience in secure or restricted environments Proven delivery of data pipelines using Prefect or Airflow Hands-on experience with Docker, Kubernetes , and container orchestration Experience operating in air-gapped systems , managing builds without external connectivity Strong understanding of data ...

Senior DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience ...

DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience Hands ...