1 to 25 of 60 Permanent Apache Airflow Jobs in the UK

What you'll be doing: * Building and maintaining scalable data pipelines and workflows * Developing Python based solutions for data processing and integration * Working with Apache Airflow to orchestrate data workflows * Supporting containerised environments using Docker * Collaborating with engineers, analysts, and client teams on technical delivery * Contributing to continuous … improvement and platform reliability What you'll bring: * Strong hands on experience with Python 3 * Experience working with Apache Airflow * Experience working within Dockerised environments * Good understanding of data engineering principles and ETL processes * Strong problem solving and communication skills * Financial markets or trading domain exposure would ...

What you'll be doing: * Building and maintaining scalable data pipelines and workflows * Developing Python based solutions for data processing and integration * Working with Apache Airflow to orchestrate data workflows * Supporting containerised environments using Docker * Collaborating with engineers, analysts, and client teams on technical delivery * Contributing to continuous … improvement and platform reliability What you'll bring: * Strong hands on experience with Python 3 * Experience working with Apache Airflow * Experience working within Dockerised environments * Good understanding of data engineering principles and ETL processes * Strong problem solving and communication skills * Financial markets or trading domain exposure would ...

automate workflows, and deliver secure, high-performance data platforms using AWS-native technologies. Key Responsibilities Develop and maintain data pipelines using Python , SQL , and Apache Airflow . Build and manage containerised data applications using Docker . Design and optimise solutions using AWS services including Redshift, OpenSearch, Lambda, Glue … data governance, security, and best practice standards. Essential Skills Active Security Clearance Strong experience with Python and SQL . Hands-on experience with Apache Airflow and Docker . Proven experience with AWS data services, including Redshift, OpenSearch, Lambda, Glue, Step Functions, and Batch . Experience with Git , GitLab ...

environment. You'll have the opportunity to contribute to AI initiatives, support evolving data platforms, and work with emerging technologies including Microsoft Fabric, GitHub, Apache Airflow, and intelligent automation tooling. The team offers a supportive culture, strong learning opportunities, and clear pathways for career progression and technical growth. … analytics teams with high-quality data engineering practices. Contribute to strategic platform modernisation initiatives including AI projects, Microsoft Fabric exploration, GitHub adoption, and Apache Airflow implementation. Work within a collaborative Azure-based data engineering environment, bringing strong SQL and general data engineering expertise; SAS experience is beneficial ...

career in data engineering, it can feel like youre staring at a never-ending list of tools and technologies SQL, Python, Spark, Kafka, Airflow, dbt, Snowflake, Redshift, Terraform, Kubernetes, and the list goes on. Scroll job boards and LinkedIn, and its easy to conclude that unless you have experience ...

career in data engineering, it can feel like youre staring at a never-ending list of tools and technologies SQL, Python, Spark, Kafka, Airflow, dbt, Snowflake, Redshift, Terraform, Kubernetes, and the list goes on. Scroll job boards and LinkedIn, and its easy to conclude that unless you have experience ...

Data Engineer: Passion for building scalable, resilient cloud data platforms Strong experience with Databricks or Snowflake on AWS Proven Python skills, including Spark and Airflow expertise Advanced SQL skills and end-to-end data modelling experience Experience building batch and real-time data integrations Hands-on CICD skills with ...

Glue, ECR, EC2, EMR, DynamoDB, Lake Formation Experience with Infrastructure as Code (Terraform) Proficiency in Python, SQL, and familiarity with PySpark Experience with Apache Spark and data processing frameworks Hands-on experience with Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/or Amazon Athena ...

technical stakeholders Strong communication, problem-solving and analytical skills Desirable Experience Experience with Azure, AWS or GCP Exposure to orchestration tools such as Airflow, Dagster or Fabric Pipelines Experience with Git, CI/CD pipelines and DataOps practices Knowledge of Microsoft Purview, Unity Catalog or similar governance tools Experience ...

model on hybrid/on-prem environments. Hard Skills Experience in data engineering, ideally aerospace/defence or industrial sectors. ETL/ELT tools (Airflow, Azure, Microsoft SSIS), streaming (Spark, Kafka). Strong Python and SQL; C++, C#, or MATLAB a plus. Databases: PostgreSQL, SQL Server, Oracle, Snowflake, Iceberg ...

Spark, Databricks, Hadoop or cloud-native alternatives). Experience with containerisation and DevOps/MLOps tools (e.g. Docker, Kubernetes, Git, CI/CD, MLflow, Airflow). Strong experience with data engineering concepts (ETL/ELT, data integration, warehousing, vector databases/search) A little bit more about us... ...

/EKS, Step Functions). Experience with MLOps tooling such as: Experiment tracking and model registries (MLflow, Weights & Biases, SageMaker Model Registry) Workflow orchestration (Airflow, Prefect, Step Functions) Model serving frameworks (SageMaker, TorchServe, TensorFlow Serving, or similar) Feature stores (Feast, Tecton, or custom-built) Deep understanding of monitoring ...

analytics and reporting, and contributing to the adoption of AI-powered workflows and automation. Key Responsibilities & Skills: Build and maintain scalable data pipelines using Apache Airflow or similar orchestration tools. Develop integrations with internal and third-party APIs, ensuring reliable and efficient data flows. Design and optimise ...

ensuring data integrity across every layer Lead the transformation of legacy reporting (Python, SQL, Excel) into a secure,scalableand fully observable platform powered by Apache Airflow (WMAA). Define and publish data quality metrics that drive engineering decisions anddemonstrateplatform health to stakeholders Champion AI and modern data tooling ...

with a strong focus on ETL/ELT. • Advanced SQL and Python development skills. • Hands-on experience with DBT, GIT, Terraform, Docker, IAM, and Airflow (Composer). • Proven experience working on cloud platforms – ideally GCP (BigQuery), but Azure or AWS also considered. • Strong understanding of Data Mesh, Test Driven ...

including optimisation of complex queries (E). Experience building data pipelines and ETL/ELT workflows using tools such as: Azure Data Factory, Databricks, Airflow, Luigi, or similar (E) Strong understanding of data modelling (E) Programming skills in Python and/or Scala for data processing (D). Experience ...

/T-SQL with proven experience refactoring and optimising complex logic Hands-on experience with cloud data warehouses (e.g. BigQuery, Snowflake), ingestion tools (Airflow, Airbyte), and analytics tooling such as dbt and Power BI Strong working knowledge of Git/version control, CI/CD pipelines, and Python ...

preparation specifically geared toward AI model integration Technical Fluency: A strong conceptual understanding of modern data ecosystems, data warehousing, pipeline orchestration tools (e.g., dbt, Airflow), and data governance standards Cross-Functional Communication: Outstanding stakeholder management skills, with a demonstrated ability to foster collaboration between technical developers and non-technical ...

Redux, Redux Hooks, and other modern state management libraries. Kotlin on Android and Swift on iOS. Data Analytics/Engineering: SQL, Python, Snowflake, dbt, Airflow & Looker. AI at Marshmallow: Sierra.ai for conversational AI, Python and LangChain for agentic AI solutions, SageMaker and MLflow for Machine Learning, Cursor and Claude ...

Proven experience in a Lead/Senior Data Engineering role Strong handson experience with Snowflake Expertise in ETL/ELT and orchestration tools (e.g. Airflow, dbt, Matillion) Strong SQL and Python skills Experience building scalable data platforms in cloud environments Previous experience leading or mentoring engineers Strong communication ...

warehousing concepts Exposure to data transformation tools such as DBT is beneficial Basic knowledge of Python, ideally in the context of orchestration tools like Airflow Confident communication skills and the ability to work effectively in a remote team A proactive approach with the ability to take ownership and solve ...

production-grade data pipelines Solid understanding of integration patterns including REST APIs, webhooks, OAuth and event-driven architectures Experience with orchestration tools such as Airflow, Prefect or Dagster Familiarity with cloud environments (AWS, GCP or Azure) Experience with containerisation tools such as Docker and Kubernetes Proven experience integrating multiple ...

necessary infrastructure, resources, and interfaces to enable data loading and LLM workflows. Use Python/Java and large-scale data workflow orchestration platforms e.g. Airflow to construct software artifacts for ETL, interfacing with diverse data formats and storage technologies, and incorporate them into robust data workflows and dynamic systems ...

necessary infrastructure, resources, and interfaces to enable data loading and LLM workflows. Use Python/Java and large-scale data workflow orchestration platforms e.g. Airflow to construct software artifacts for ETL, interfacing with diverse data formats and storage technologies, and incorporate them into robust data workflows and dynamic systems ...

common CI/CD stack (e.g., Jenkins, CircleCI, GitLab, Azure DevOps) Experience in workflow orchestration with tools such as Argo Workflow, Airflow, and scientific workflow tools such as Nextflow, Snakemake, VisTrails, or Cromwell Experience with application performance tuning and optimization, including in parallel and distributed computing paradigms and communication ...