26 to 50 of 103 Apache Airflow Jobs in the UK excluding London

writing automation scripts or custom AI agents using node or python. Data Engineering (30%): Experience deploying and maintaining data orchestration toolsets (e.g., dbt, Apache Airflow), ETL/ELT methodologies, real‐time streaming (e.g., AWS Kinesis, Apache Kafka), Vector databases, and RAG architectures. Observability & FinOps: Experience implementing modern ...

writing automation scripts or custom AI agents using node or python. Data Engineering (30%): Experience deploying and maintaining data orchestration toolsets (e.g., dbt, Apache Airflow), ETL/ELT methodologies, real‐time streaming (e.g., AWS Kinesis, Apache Kafka), Vector databases, and RAG architectures. Observability & FinOps: Experience implementing modern ...

writing automation scripts or custom AI agents using node or python. Data Engineering (30%): Experience deploying and maintaining data orchestration toolsets (e.g., dbt, Apache Airflow), ETL/ELT methodologies, real‐time streaming (e.g., AWS Kinesis, Apache Kafka), Vector databases, and RAG architectures. Observability & FinOps: Experience implementing modern ...

/T-SQL with proven experience refactoring and optimising complex logic Hands-on experience with cloud data warehouses (e.g. BigQuery, Snowflake), ingestion tools (Airflow, Airbyte), and analytics tooling such as dbt and Power BI Strong working knowledge of Git/version control, CI/CD pipelines, and Python ...

supporting Spark‐based platforms (e.g. EMR, Glue, Databricks) (Preferred) Proficient with DevOps tools and practices (Git, Jenkins, Docker, Kubernetes) as well as ETL frameworks (Apache Spark, Airflow, AWS IoT Analytics/Device Management/Events). (Preferred) Programming expertise in Python, Java, Scala, SQL, Spark, Unix shell scripting ...

supporting Spark‐based platforms (e.g. EMR, Glue, Databricks) (Preferred) Proficient with DevOps tools and practices (Git, Jenkins, Docker, Kubernetes) as well as ETL frameworks (Apache Spark, Airflow, AWS IoT Analytics/Device Management/Events). (Preferred) Programming expertise in Python, Java, Scala, SQL, Spark, Unix shell scripting ...

initial 6-month contract. Core Technical Requirements Strong experience with Snowflake for data warehousing, including writing efficient SQL and managing schemas. Proficiency in Airflow for orchestration and workflow management. Hands-on experience with AWS services , particularly S3 for storage and Lambda for serverless processing. Solid understanding of data modelling ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing vector databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing vector databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing vector databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

implementing agentic guardrails. Data Engineering (30%): Practical experience implementing components of data pipelines, including real-time streaming tools (AWS Kinesis, Kafka), data orchestration (dbt, Airflow), and managing Vector Databases for RAG architectures. Observability & Cost Management: SRE/Platform experience with the practical application of real-time monitoring and cloud ...

Application Resiliency, and Security. Preferred qualifications, capabilities, and skills Production experience with Databricks (Delta Lake, Unity Catalog, MLflow, Databricks Jobs) and workflow orchestration with Apache Airflow. Strong working knowledge of both supervised and unsupervised techniques, including tree-based and kernel methods (gradient boosting, random forest, SVM) and anomaly-detection ...

required . Proficiency in at least one object-oriented language. Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto. Experience with AWS, GCP, or Looker (advantageous but not essential). Solid background in data modelling, including the design, implementation and optimisation ...

required . Proficiency in at least one object-oriented language. Experience with big data technologies such as HDFS, YARN, MapReduce, Hive, Kafka, Spark, Airflow, or Presto. Experience with AWS, GCP, or Looker (advantageous but not essential). Solid background in data modelling, including the design, implementation and optimisation ...

Comfortable building from scratch in a fast-moving, product-led environment Experience with vector databases and vector search in modern AI systems Desirable: Spark, Airflow, Kafka, Elasticsearch/OpenSearch, event-driven architectures Cloud experience (AWS, GCP or Azure), GCP is a plus Why join High impact and ownership ...

degree in Computer Science, Engineering, or a related technical field, or equivalent practical experience. Preferred Skills Experience with data orchestration tools (e.g., Prefect, Airflow, dbt). Familiarity with MLOps tooling such as MLflow, or experience supporting ML workloads in production. Exposure to LLM or agentic application patterns ...

delivery (CI/CD) pipelines, containerisation, and workflow orchestration. Familiar with ETL/ELT frameworks, and experienced with Big Data Processing Tools (e.g. Spark, Airflow, Hive, etc.) Knowledge of programming languages (e.g. Java, Python, SQL) Hands-on experience with SQL/NoSQL database design. Degree in STEM, or similar ...

delivery (CI/CD) pipelines, containerisation, and workflow orchestration. Familiar with ETL/ELT frameworks, and experienced with Big Data Processing Tools (e.g. Spark, Airflow, Hive, etc.) Knowledge of programming languages (e.g. Java, Python, SQL) Hands-on experience with SQL/NoSQL database design. Degree in STEM, or similar ...

delivery (CI/CD) pipelines, containerisation, and workflow orchestration. Familiar with ETL/ELT frameworks, and experienced with Big Data Processing Tools (e.g. Spark, Airflow, Hive, etc.) Knowledge of programming languages (e.g. Java, Python, SQL) Hands-on experience with SQL/NoSQL database design. Degree in STEM, or similar ...

practical experience. Preferred Skills Experience with gradient-boosted tree models, neural networks, and optimization solvers in production. Familiarity with data orchestration tools (e.g., Prefect, Airflow, dbt) and modern data lake architectures. Experience working closely with data scientists to operationalize research code. Multi-cloud experience across AWS, GCP, and Azure. ...

Working knowledge of Kubernetes/EKS, comfortable deploying to and reasoning about containerised workloads in a managed K8s environment. Experience orchestrating pipelines with Dagster, Airflow, Prefect, or similar. Solid grasp of ETL concepts and production‐grade pipeline design. Beneficial Skills Interest in chemistry, life sciences, robotics, or manufacturing. Hands ...

data platform. You'll play a key role in building and optimising scalable data pipelines and analytics solutions, working across AWS, Snowflake, dbt and Airflow to deliver high-quality data products. The Company They are a well-established media organisation with a large and diverse audience, using data … models. Your Skills & Experience Strong experience with SQL and modern data engineering practices. Hands-on experience with Snowflake and dbt. Strong experience working with Apache Airflow for data orchestration. Experience working with AWS data services and cloud-based data platforms. Strong Python skills for data engineering and pipeline ...

code Requirements: - Must have strong, commercial AWS (contract) Data Engineering experience - Must bring solid knowledge of data warehousing and modelling - Experience with Airflow and infrastructure as a code (Terraform, CloudFormation) would be beneficial - Must have experience with Redshift Nice to have: - Experience with dbt/lake formation ...

notebooks for analysis dbt and Databricks data platform with Delta Lake Lightdash as our BI tool and semantic layer Streamlit for interactive data applications Airflow for orchestration Kubernetes for application deployment CI/CD with GitHub and CircleCI Infrastructure on AWS, deployed with Terraform/Spacelift ...

Arrow, plus an analytical engine, e.g. DuckDB, or a warehouse such as Snowflake. Orchestrated batch pipelines you've operated, not just written: Dagster or Airflow, S3-based data flows, and a habit of testing outputs for correctness rather than only for exceptions. Applied ML in production: embeddings and semantic ...