1 to 25 of 45 Apache Airflow Jobs in the South East

services, including microservice deployment Developing and maintaining data pipelines to process and manage large-scale structural biology data Working with workflow orchestration tools like Apache Airflow and Nextflow To liaise with the EMBL-EBI Technical Services Cluster To support the development and deployment of other scientific software The … Proficiency in containerisation Proficiency in relational databases (Oracle, PostgreSQL) Experience in developing and maintaining data pipelines Hands-on experience with workflow orchestration tools (e.g., Apache Airflow, Nextflow) Strong interpersonal and communication skills Proficiency in oral and written English You may also have PhD in computer science, IT or … a related field, or in bioinformatics with a demonstrated IT expertise Experience in using Kubernetes Experience with web servers (Apache/Nginx) Hands-on experience with CI/CD (GitLab CI/GitHub Actions) Familiarity with networking Familiarity with Java Knowledge of, or affinity with, structural biology and bioinformatics More ❯

AWS preferred) Solid understanding of libraries like Pandas and NumPy Experience in data warehousing tools like Snowflake, Databricks, BigQuery Familiar with AWS Step Functions, Airflow, Dagster, or other workflow orchestration tools Commercial experience with performant database programming in SQL Capability to solve complex technical issues, comprehending risks prior to More ❯

as Redshift, Snowflake, or BigQuery. Strong command of SQL and programming languages like Python, Scala, or Java. Familiarity with ETL/ELT tools (e.g., Airflow, Fivetran, dbt) and cloud data stacks (AWS/GCP/Azure). A deep understanding of data modelling, access controls, and infrastructure performance tuning. More ❯

Have touched cloud services (AWS, GCP, etc.) Enjoy solving problems, learning fast, and working with good people Bonus points if you’ve played with Airflow, Docker, or big data tools — but they’re more interested in mindset than buzzwords. The team’s based onsite in London — they’re collaborative More ❯

transformation, cleaning, and loading. Strong coding experience with Python and Pandas. Experience with any flavour of data pipeline and workflow management tools: Dagster, Celery, Airflow, etc. Build processes supporting data transformation, data structures, metadata, dependency and workload management. Experience supporting and working with cross-functional teams in a dynamic More ❯

and storage. Strong programming skills in Python, Java, or Scala . Proficiency in SQL, NoSQL, and time-series databases . Knowledge of orchestration tools (Apache Airflow, Kubernetes). If you are a passionate and experienced Senior Data Engineer seeking a Lead role, or a Lead Data Engineer aiming More ❯

SQL and experienced with various database technologies. Knowledge of Python or Java, with the ability to leverage either in building scalable solutions. Experience with Airflow & other big data technologies is useful. Familiarity with DevOps practices and tools, including CI/CD pipelines. Previous experience with reporting tools is helpful More ❯

functional teams Tech You'll Work With ML & Data Science Python (primary language) TensorFlow, PyTorch, or Keras NumPy, pandas Data pipelines (Azure Data Factory, Airflow, etc.) Applied ML: NLP, CV, transformers, GANs, time series, etc. Engineering & Cloud Azure (or similar cloud platforms like AWS, GCP) Microservices and event-driven More ❯

functional teams Tech You'll Work With ML & Data Science Python (primary language) TensorFlow, PyTorch, or Keras NumPy, pandas Data pipelines (Azure Data Factory, Airflow, etc.) Applied ML: NLP, CV, transformers, GANs, time series, etc. Engineering & Cloud Azure (or similar cloud platforms like AWS, GCP) Microservices and event-driven More ❯

field. Proficiency in programming languages such as Python, Spark, SQL. Strong experience with SQL databases. Expertise in data pipeline and workflow management tools (e.g., Apache Airflow, ADF). Experience with cloud platforms (Azure preferred) and related data services. There’s no place quite like BFS and we’re More ❯

field. Proficiency in programming languages such as Python, Spark, SQL. Strong experience with SQL databases. Expertise in data pipeline and workflow management tools (e.g., Apache Airflow, ADF). Experience with cloud platforms (Azure preferred) and related data services. There's no place quite like BFS and we're More ❯

orchestration Contribute to scalable, governed AI/ML-ready data environments Tech Stack: GitHub Actions/GitLab CI/Jenkins | Terraform | AWS | Snowflake | dbt | Airflow | Python/Bash | Docker/Kubernetes (nice to have) Requirements: Proven DevOps/Platform Engineer in data-centric settings CI/CD & IaC (Terraform More ❯

business needs into technical solutions using SQL and cloud tools. 🧠 What you’ll bring: Strong SQL skills Proven experience building and maintaining data pipelines (Airflow, DBT, etc.). Familiarity with cloud data platforms like Snowflake, BigQuery, or Redshift. Solid experience with BI tools like Looker, Tableau, or similar. Understanding More ❯

coding practices. Results-driven, with the capacity to meet strict deadlines and service level agreements (SLAs). Ideally, familiarity with workflow management tools like Airflow, SQL database management, Linux command-line interface, Microsoft Dynamics 365 Finance and Operations, and Power Apps. The Package: Basic annual salary up to More ❯

related field. 5+ years of experience in data engineering and data quality. Strong proficiency in Python/Java, SQL, and data processing frameworks including Apache Spark. Knowledge of machine learning and its data requirements. Attention to detail and a strong commitment to data integrity. Excellent problem-solving skills and More ❯

to enhance scalability and efficiency. Collaboration & Leadership : Work closely with software and AI engineering teams while mentoring junior engineers. Legacy Workflow Integration : Manage ArgoCD, Airflow, Jenkins, Bitbucket, and Bamboo pipelines. Technical Ownership : Act as a tech owner for software products, liaising with stakeholders and presenting cloud solutions. Continuous Learning More ❯

data platform (PaaS) Ensure scalable storage solutions (data lakes, data warehouses) to handle structured and unstructured data. Implement ETL/ELT pipelines using Dagster, Airflow, or similar tools. Optimize performance and scalability for large data volumes. Govern data security, compliance, and access controls. Development & DevOps: Strong programming and scripting More ❯

in data pipelines across cloud/on-premises, using Azure and other technologies. Experienced in orchestrating data workflows and Kubernetes clusters on AKS using Airflow, Kubeflow, Argo, Dagster or similar. Skilled with data ingestion tools like Airbyte, Fivetran, etc. for diverse data sources. Expert in large-scale data processing More ❯

tuning and optimisation Solid understanding of data warehousing principles, data modelling practice, Excellent knowledge of creation and maintenance of data pipelines - ETL Tools (e.g. Apache Airflow) and Streaming processing tools (e.g. Kinesis) Strong problem-solving and analytical skills, with the ability to troubleshoot and resolve complex data-related More ❯

tuning and optimisation ·Solid understanding of data warehousing principles, data modelling practice, ·Excellent knowledge of creation and maintenance of data pipelines - ETL Tools (e.g. Apache Airflow) and Streaming processing tools (e.g. Kinesis) ·Strong problem-solving and analytical skills, with the ability to troubleshoot and resolve complex data-related More ❯

tuning and optimisation Solid understanding of data warehousing principles, data modelling practice, Excellent knowledge of creation and maintenance of data pipelines - ETL Tools (e.g. Apache Airflow) and Streaming processing tools (e.g. Kinesis) Strong problem-solving and analytical skills, with the ability to troubleshoot and resolve complex data-related More ❯

commodities markets (e.g., oil, gas). The PM is keen to get someone from a data engineering background. Tech: Python, SQL, Pandas, AWS, ETL, Airflow Please apply if this fits. More ❯

SQL expertise designing and maintaining ETL/data pipelines ideally proficiency in multiple cloud infrastructures, databases and data warehousing solutions - AWS and GCP being Airflow 👍 Bonus points for: Experience working in a small, fast-growing start-up, comfortable navigating unstructured/fuzzy environments Experience with RudderStack, Expo and/ More ❯

modelling, and extracting value from large, disconnected datasets Familiarity with data warehouse design and modelling Familiarity with cloud data services (preferably Azure) Experience with Airflow 🛂 Please note : unfortunately, this role does not offer VISA sponsorship. More ❯

So they offer very flexible working arrangements through both WFH options and flexi working hours. Experience required... Expert in Snowflake Strong DBT experience Strong Airflow experience Expert knowledge and understanding of Data Warehousing Strong AWS experience More ❯