26 to 42 of 42 Apache Airflow Jobs in London

analytical data modeling in BigQuery using dbt . Build reliable data pipelines utilizing SQL, Python, and distributed processing frameworks (Apache Spark, Scio, Beam, or Flink). Develop clear, story-driven dashboards using tools like Looker or Tableau . Champion data quality , implement CI/CD for dbt models … years of experience in analytics or data engineering, with deep expertise in SQL. Extensive experience with dbt, a cloud data warehouse , and workflow orchestrators (Airflow, Dagster, Prefect, or Flyte). Proficiency in Python for data analysis and automation. Bonus: Experience with experimentation, ML/AI metrics, or platform productivity ...

Python, and Ansible Familiarity with containers (Docker, Kubernetes, LXC) Experience with monitoring tools and SQL databases Exposure to big data systems (e.g., Spark, HDFS, Airflow) Summary A great opportunity for someone who enjoys solving complex infrastructure challenges, automating systems, and working closely with engineering teams. Spectrum IT Recruitment (South ...

Python, and Ansible Familiarity with containers (Docker, Kubernetes, LXC) Experience with monitoring tools and SQL databases Exposure to big data systems (e.g., Spark, HDFS, Airflow) Summary A great opportunity for someone who enjoys solving complex infrastructure challenges, automating systems, and working closely with engineering teams. Spectrum IT Recruitment (South ...

moving away from legacy systems towards a scalable, cloud-native stack. What you'll be working on: Building and maintaining scalable data pipelines using Airflow and dbt Working across AWS cloud infrastructure Collaborating with analysts and data scientists to deliver reliable, clean data for downstream reporting and business decision … governance, and best practices across the team What they're looking for: Proven data engineering experience with strong Python and SQL Hands-on with Airflow and/or dbt Solid AWS experience Experience with ETL pipeline development at scale Comfortable working in Agile/Scrum environments Full right ...

ingestion workflows Implement data quality frameworks and enforce data contracts Build CI/CD pipelines and automated testing for data workflows Orchestrate pipelines using Airflow, Prefect, or Dagster Improve observability, monitoring, and lineage across data systems Mentor junior engineers and elevate engineering standards Key Skills and Experience Management … Data Engineering experience Expert-level SQL and advanced data modelling Strong Python Experience with Snowflake and Databricks Cloud experience Experience with orchestration tools (Airflow, Prefect) CI/CD and DataOps experience for data platforms This role will suit someone who is looking to remain hands on , whilst leading ...

Platforms Itential Automation Platform Cisco NSO – Service development and lifecycle Apache Airflow Hasi Corp Vault Openshift Automation development Python Fast API Rest API Netconf Yang modelling XML/Json Jinja/TextFSM Pytest Unit testing Database MongoDB DevOps Docker Git CI/CD Kubernetes Networking Ciena 517x/ ...

including managing dependencies and build pipelines without internet access Proven background building APIs for data enrichment services Orchestration & data pipeline experience with Prefect or Airflow, and K8s or Swarm Configuration and automation of business workflow tooling (e.g., Jira) Hands-on experience with a range of databases and storage technologies ...

including managing dependencies and build pipelines without internet access Proven background building APIs for data enrichment services Orchestration & data pipeline experience with Prefect or Airflow, and K8s or Swarm Configuration and automation of business workflow tooling (e.g., Jira) Hands-on experience with a range of databases and storage technologies ...

+ delivery). Strong command of APIs, event-driven patterns, queues, webhooks and orchestration methods. Hands on with Azure APIM, MuleSoft, Boomi, WSO2, Kafka, Airflow or similar. Understanding of data lineage, transformation and complex data flows. Enterprise security, IAM and API governance expertise. (Desirable) Exposure to Lloyd ...

warehousing concepts Familiarity with version control systems, particularly Git Desirable Skills: Experience with infrastructure as code tools such as Terraform or CloudFormation Exposure to Apache Spark for distributed data processing Familiarity with workflow orchestration tools such as Airflow or AWS Step Functions Understanding of containerisation using Docker Experience ...

root cause analysis, managing SLAs, and improving resiliency across business-critical data feeds. Requirements: SQL Python/Bash ETL/ELT support Workflow orchestration (Airflow or Dagster) AWS data services (Athena, S3, MWAA, Iceberg) Data quality monitoring & incident management Nice to Have: Airbyte DBT Apache Spark Grafana ...

Engineer (Mid Level) £45,000–£55,000 Remote (1-2 days per month London, Leeds or Preston) Tech Stack: GCP (BigQuery, Dataflow), DBT, Terraform, Airflow (Composer), Python, SQL VIQU are working with a growing, data-driven organisation they are building out a modern Data Platform following a successful migration … pipelines Strong SQL and Python skills, with hands-on experience building data pipelines Experience with DBT, Terraform, and version control (Git) Exposure to Airflow (or similar orchestration tools), Docker, and CI/CD practices Cloud experience in GCP (BigQuery preferred) or Azure/AWS environments Understanding of modern data ...

Engineer (Mid Level) £45,000–£55,000 Remote (1-2 days per month London, Leeds or Preston) Tech Stack: GCP (BigQuery, Dataflow), DBT, Terraform, Airflow (Composer), Python, SQL VIQU are working with a growing, data-driven organisation they are building out a modern Data Platform following a successful migration … pipelines Strong SQL and Python skills, with hands-on experience building data pipelines Experience with DBT, Terraform, and version control (Git) Exposure to Airflow (or similar orchestration tools), Docker, and CI/CD practices Cloud experience in GCP (BigQuery preferred) or Azure/AWS environments Understanding of modern data ...

Backend: Scala, Java (Python desirable) Frontend: HTML, CSS, JavaScript Frameworks: Play, Spark Data: Postgres, MongoDB, Hudi Cloud: AWS (S3, ECS, Aurora, Lambda) DevOps: Jenkins, Airflow, Terraform Monitoring: Kibana, Grafana Ways of working: Agile (Scrum/Kanban), CI/CD, 'you build it, you run it' What We're Looking … modern DevOps practices Experience working in Agile, multidisciplinary teams Strong analytical and problem-solving skills Clear communicator and collaborative team player Desirable Python, Docker, Airflow Big data tools (e.g. Spark, EMR) Experience with data platforms or risk/fraud systems How You'll Work Deliver value iteratively and early ...

security. Drive modernisation by transitioning from legacy systems to a lean, scalable platform. Act as a lead expert for technologies such as AWS, DBT, Airflow, and Databricks. Establish best practices for data modelling, ingestion, storage, streaming, and APIs. Governance & Standards Ensure all technical decisions are well-justified, documented … engineering, including data ingestion, transformation, and storage. Significant hands-on experience with AWS and its data services. Expert-level skills in SQL, Python, DBT, Airflow and Redshift. Confidence in coding, scripting, configuring, versioning, debugging, testing, and deploying. Ability to guide and mentor others in technical best practices. A product ...

financial services organisation Your new role We are seeking a Data Engineer to support the replacement of a legacy ETL tool with a modern Apache Spark based data platform. This is a hands-on engineering role focused on building and supporting Spark jobs, with an emphasis on performance, reliability … scalability.The role is focused on building nonperformance Apache Spark jobs, with a strong emphasis on performance optimisation. It is primarily Java-based, with Python strongly encouraged; experience across Java, Python, and Scala is ideal, with Java and Python being the preferred combination.The role sits within a small Agile delivery ...

Senior DataEngineer-RemoteWorking LynxarecurrentlyworkingwithalargeITconsultancytohelpthemsourceaDataEngineer.OurclientislargeglobalConsultancywhoworkwithenterpriseclients,solvingtheirbusinessandtechnologyproblemsusingcuttingedgesolutions. Asa DataEngineer ,youwillplayakeyroleindesigning,building,andmaintainingrobustdataplatformsandpipelines.Youllworkalongsidedatascientists,analysts,andengineerstoensurereliable,scalable,andhigh-qualitydatasolutions. Whatyoullbedoing: Designingandbuildingdatapipelines,models,andarchitectures DevelopingandmaintainingETL/ELTprocessesusingmoderntools Implementingdatastoragesolutionsacrossrelational,non-relational,andcloudplatforms Ensuringdataquality,reliability,andoperationalexcellence Leadingthedesignanddeliveryofnewdataproductsandpipelines Collaboratingcloselywithstakeholdersandtechnicalteams Providingtechnicalleadershipandmentoringwithindeliveryteams Whattherelookingfor: Strongexperienceindataengineeringroles Provenleadershiporlinemanagementexperience ...