1 to 25 of 52 Apache Airflow Jobs in the UK excluding London

What you'll be doing: * Building and maintaining scalable data pipelines and workflows * Developing Python based solutions for data processing and integration * Working with Apache Airflow to orchestrate data workflows * Supporting containerised environments using Docker * Collaborating with engineers, analysts, and client teams on technical delivery * Contributing to continuous … improvement and platform reliability What you'll bring: * Strong hands on experience with Python 3 * Experience working with Apache Airflow * Experience working within Dockerised environments * Good understanding of data engineering principles and ETL processes * Strong problem solving and communication skills * Financial markets or trading domain exposure would ...

What you'll be doing: * Building and maintaining scalable data pipelines and workflows * Developing Python based solutions for data processing and integration * Working with Apache Airflow to orchestrate data workflows * Supporting containerised environments using Docker * Collaborating with engineers, analysts, and client teams on technical delivery * Contributing to continuous … improvement and platform reliability What you'll bring: * Strong hands on experience with Python 3 * Experience working with Apache Airflow * Experience working within Dockerised environments * Good understanding of data engineering principles and ETL processes * Strong problem solving and communication skills * Financial markets or trading domain exposure would ...

professionals with strong Python development expertise, cloud engineering experience, and exposure to modern DevOps and data engineering practices. Must Have: AWS services; Terraform; Apache Spark; Python; Apache Airflow; Docker; GitLab; Security scanning agent; Jupyter; Notebooks; Key Responsibilities Design, develop, and maintain scalable Python applications and services Build … optimize cloud-native data processing pipelines Develop and manage workflows using orchestration tools such as Airflow Work with large-scale data processing frameworks including Apache Spark Deploy and maintain containerized applications using Docker Implement infrastructure automation using Terraform Integrate solutions with enterprise data and secure file transfer systems ...

data validation and application testing processes across distributed systems Investigate production issues using data analysis to identify root causes Work with orchestration tools (e.g., Apache Airflow) to monitor workflows and troubleshoot failures Validate data using tools such as Jupyter Notebooks and Amazon Athena Analyse data across multiple formats … additional programming language (e.g., Java) Experience with scripting languages such as Bash Strong understanding of automated testing frameworks and methodologies Hands-on experience with Apache Airflow and log analysis Familiarity with data processing tools such as Apache Spark Experience working with cloud platforms and services, including: Monitoring ...

automate workflows, and deliver secure, high-performance data platforms using AWS-native technologies. Key Responsibilities Develop and maintain data pipelines using Python , SQL , and Apache Airflow . Build and manage containerised data applications using Docker . Design and optimise solutions using AWS services including Redshift, OpenSearch, Lambda, Glue … data governance, security, and best practice standards. Essential Skills Active Security Clearance Strong experience with Python and SQL . Hands-on experience with Apache Airflow and Docker . Proven experience with AWS data services, including Redshift, OpenSearch, Lambda, Glue, Step Functions, and Batch . Experience with Git , GitLab ...

environment. You'll have the opportunity to contribute to AI initiatives, support evolving data platforms, and work with emerging technologies including Microsoft Fabric, GitHub, Apache Airflow, and intelligent automation tooling. The team offers a supportive culture, strong learning opportunities, and clear pathways for career progression and technical growth. … analytics teams with high-quality data engineering practices. Contribute to strategic platform modernisation initiatives including AI projects, Microsoft Fabric exploration, GitHub adoption, and Apache Airflow implementation. Work within a collaborative Azure-based data engineering environment, bringing strong SQL and general data engineering expertise; SAS experience is beneficial ...

Design, build, and maintain cloud-based infrastructure using AWS Develop and manage Infrastructure as Code using Terraform Support and optimise data processing pipelines using Apache Spark and Airflow Collaborate with cross-functional teams to deliver secure and scalable solutions Implement CI/CD pipelines using GitLab Containerise applications … Notebooks for data exploration and development Key Skills & Experience Strong experience with AWS services Expertise in Terraform (Infrastructure as Code) Hands-on experience with: Apache Spark Python Apache Airflow Docker GitLab CI/CD Experience with security scanning tools such as: Trivy Trend Micro Wiz (or similar ...

career in data engineering, it can feel like youre staring at a never-ending list of tools and technologies SQL, Python, Spark, Kafka, Airflow, dbt, Snowflake, Redshift, Terraform, Kubernetes, and the list goes on. Scroll job boards and LinkedIn, and its easy to conclude that unless you have experience ...

career in data engineering, it can feel like youre staring at a never-ending list of tools and technologies SQL, Python, Spark, Kafka, Airflow, dbt, Snowflake, Redshift, Terraform, Kubernetes, and the list goes on. Scroll job boards and LinkedIn, and its easy to conclude that unless you have experience ...

Data Engineer: Passion for building scalable, resilient cloud data platforms Strong experience with Databricks or Snowflake on AWS Proven Python skills, including Spark and Airflow expertise Advanced SQL skills and end-to-end data modelling experience Experience building batch and real-time data integrations Hands-on CICD skills with ...

Glue, ECR, EC2, EMR, DynamoDB, Lake Formation Experience with Infrastructure as Code (Terraform) Proficiency in Python, SQL, and familiarity with PySpark Experience with Apache Spark and data processing frameworks Hands-on experience with Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/or Amazon Athena ...

technical stakeholders Strong communication, problem-solving and analytical skills Desirable Experience Experience with Azure, AWS or GCP Exposure to orchestration tools such as Airflow, Dagster or Fabric Pipelines Experience with Git, CI/CD pipelines and DataOps practices Knowledge of Microsoft Purview, Unity Catalog or similar governance tools Experience ...

model on hybrid/on-prem environments. Hard Skills Experience in data engineering, ideally aerospace/defence or industrial sectors. ETL/ELT tools (Airflow, Azure, Microsoft SSIS), streaming (Spark, Kafka). Strong Python and SQL; C++, C#, or MATLAB a plus. Databases: PostgreSQL, SQL Server, Oracle, Snowflake, Iceberg ...

Spark, Databricks, Hadoop or cloud-native alternatives). Experience with containerisation and DevOps/MLOps tools (e.g. Docker, Kubernetes, Git, CI/CD, MLflow, Airflow). Strong experience with data engineering concepts (ETL/ELT, data integration, warehousing, vector databases/search) A little bit more about us... ...

analytics and reporting, and contributing to the adoption of AI-powered workflows and automation. Key Responsibilities & Skills: Build and maintain scalable data pipelines using Apache Airflow or similar orchestration tools. Develop integrations with internal and third-party APIs, ensuring reliable and efficient data flows. Design and optimise ...

ensuring data integrity across every layer Lead the transformation of legacy reporting (Python, SQL, Excel) into a secure,scalableand fully observable platform powered by Apache Airflow (WMAA). Define and publish data quality metrics that drive engineering decisions anddemonstrateplatform health to stakeholders Champion AI and modern data tooling ...

with a strong focus on ETL/ELT. • Advanced SQL and Python development skills. • Hands-on experience with DBT, GIT, Terraform, Docker, IAM, and Airflow (Composer). • Proven experience working on cloud platforms – ideally GCP (BigQuery), but Azure or AWS also considered. • Strong understanding of Data Mesh, Test Driven ...

including optimisation of complex queries (E). Experience building data pipelines and ETL/ELT workflows using tools such as: Azure Data Factory, Databricks, Airflow, Luigi, or similar (E) Strong understanding of data modelling (E) Programming skills in Python and/or Scala for data processing (D). Experience ...

/T-SQL with proven experience refactoring and optimising complex logic Hands-on experience with cloud data warehouses (e.g. BigQuery, Snowflake), ingestion tools (Airflow, Airbyte), and analytics tooling such as dbt and Power BI Strong working knowledge of Git/version control, CI/CD pipelines, and Python ...

data validation frameworks Experience of testing data platforms such as Snowflake, BigQuery, Redshift, Databricks, Azure Synapse Experience of testing pipelines in things like Airflow, Azure Data Factory, AWS Glue, Kafka Data Concepts ETL/ELT, data modelling, data warehousing, data governance The role is 100% remote, however you must ...

preparation specifically geared toward AI model integration Technical Fluency: A strong conceptual understanding of modern data ecosystems, data warehousing, pipeline orchestration tools (e.g., dbt, Airflow), and data governance standards Cross-Functional Communication: Outstanding stakeholder management skills, with a demonstrated ability to foster collaboration between technical developers and non-technical ...

Python for data engineering and strong SQL skills for data modelling and analysis Experience building and orchestrating data pipelines using tools such as Airflow or dbt within cloud-based environments Expertise in developing clear, user-friendly dashboards using tools such as Tableau or Looker Studio Deep understanding of music ...

Nice to Have: * Microsoft Azure Solutions Architect Expert* AWS Solutions Architect Professional* Databricks Certifications* CDMP Certification* Terraform, Bicep or Infrastructure-as-Code experience* dbt, Airflow or modern orchestration frameworks* Experience supporting AI, GenAI or RAG-based architectures* Experience within Financial Services, Healthcare, Life Sciences, Energy, Retail or Manufacturing Robert ...

Proven experience in a Lead/Senior Data Engineering role Strong handson experience with Snowflake Expertise in ETL/ELT and orchestration tools (e.g. Airflow, dbt, Matillion) Strong SQL and Python skills Experience building scalable data platforms in cloud environments Previous experience leading or mentoring engineers Strong communication ...

warehousing concepts Exposure to data transformation tools such as DBT is beneficial Basic knowledge of Python, ideally in the context of orchestration tools like Airflow Confident communication skills and the ability to work effectively in a remote team A proactive approach with the ability to take ownership and solve ...