26 to 50 of 141 Apache Airflow Jobs

Glue, ECR, EC2, EMR, DynamoDB, Lake Formation Experience with Infrastructure as Code (Terraform) Proficiency in Python, SQL, and familiarity with PySpark Experience with Apache Spark and data processing frameworks Hands-on experience with Apache Airflow for workflow orchestration Experience using Jupyter Notebooks and/or Amazon Athena ...

Glue or EKS Hands on experience with modern analytics stacks such as Snowflake, dbt and Iceberg Familiarity with orchestration tools such as Dagster or Airflow Strong understanding of data modelling concepts and analytics best practice A software engineering mindset, including Git, testing and CI/CD workflows What They ...

pipelines Familiarity with CI/CD, Git, Terraform, and modern engineering practices Experience working in Agile/Scrum environments Exposure to tools such as Airflow/Composer and Azure DevOps Understanding of AI/ML tooling such as Vertex AI Desirable Experience with BI/visualisation tools (e.g. Looker ...

secure handling of data including authentication, authorisation, encryption. Implement monitoring, alerting for large-scale distributed data workloads. Use orchestration tools such as Cloud Composer, Airflow or equivalent to operationalise pipelines. Collaborate with engineers, architects and SMEs to deliver stable, high-quality data products. Skills and Experience Strong programming skills ...

systems for unstructured data retrieval Exposure to cloud infrastructure (AWS, GCP, or Azure) for scalable data pipeline deployment Experience with modern data orchestration tools (Airflow, Prefect, dbt) What We Offer Direct access to senior football operations leadership, with genuine influence over recruitment decisions A greenfield build the opportunity ...

engineering applied to data workflows Solid SQL and experience building production‐grade pipelines Cloud data platforms (AWS, GCP, or Azure) Orchestration tools such as Airflow, Prefect, or Dagster Ability to work with stakeholders and simplify complex requirements A mindset focused on quality, reliability, and simplicity Comfortable owning problems ...

Glue or EKS Hands on experience with modern analytics stacks such as Snowflake, dbt and Iceberg Familiarity with orchestration tools such as Dagster or Airflow Strong understanding of data modelling concepts and analytics best practice A software engineering mindset, including Git, testing and CI/CD workflows What They ...

Glue or EKS Hands on experience with modern analytics stacks such as Snowflake, dbt and Iceberg Familiarity with orchestration tools such as Dagster or Airflow Strong understanding of data modelling concepts and analytics best practice A software engineering mindset, including Git, testing and CI/CD workflows What They ...

Glue or EKS Hands on experience with modern analytics stacks such as Snowflake, dbt and Iceberg Familiarity with orchestration tools such as Dagster or Airflow Strong understanding of data modelling concepts and analytics best practice A software engineering mindset, including Git, testing and CI/CD workflows What They ...

with a strong focus on ETL/ELT. • Advanced SQL and Python development skills. • Hands-on experience with DBT, GIT, Terraform, Docker, IAM, and Airflow (Composer). • Proven experience working on cloud platforms – ideally GCP (BigQuery), but Azure or AWS also considered. • Strong understanding of Data Mesh, Test Driven ...

databases (Pinecone, Weaviate, FAISS, Milvus) Data & Engineering Excellence Lead the evolution of modern data platforms (Databricks, Snowflake, Synapse, BigQuery) Ensure robust data engineering practices (Airflow, dbt, Prefect) and governance (Collibra, Purview) Drive scalable lakehouse architectures (Delta Lake, Iceberg, Hudi) Cloud, Infrastructure & MLOps Champion multi-cloud engineering across Azure ...

lead technical documentation, roadmaps, and architecture artifacts Bachelor's degree in Computer Science or related field Nice to Have Experience with Kafka, Spark, Airflow, dbt, Fivetran, Matillion, NiFi Experience with streaming, lakehouse, medallion, or enterprise migration initiatives Prior success expanding accounts or managing strategic customers Cloud certifications or advanced ...

lead technical documentation, roadmaps, and architecture artifacts Bachelor's degree in Computer Science or related field Nice to Have Experience with Kafka, Spark, Airflow, dbt, Fivetran, Matillion, NiFi Experience with streaming, lakehouse, medallion, or enterprise migration initiatives Prior success expanding accounts or managing strategic customers Cloud certifications or advanced ...

lead technical documentation, roadmaps, and architecture artifacts Bachelor's degree in Computer Science or related field Nice to Have Experience with Kafka, Spark, Airflow, dbt, Fivetran, Matillion, NiFi Experience with streaming, lakehouse, medallion, or enterprise migration initiatives Prior success expanding accounts or managing strategic customers Cloud certifications or advanced ...

lead technical documentation, roadmaps, and architecture artifacts Bachelor's degree in Computer Science or related field Nice to Have Experience with Kafka, Spark, Airflow, dbt, Fivetran, Matillion, NiFi Experience with streaming, lakehouse, medallion, or enterprise migration initiatives Prior success expanding accounts or managing strategic customers Cloud certifications or advanced ...

lead technical documentation, roadmaps, and architecture artifacts Bachelor's degree in Computer Science or related field Nice to Have Experience with Kafka, Spark, Airflow, dbt, Fivetran, Matillion, NiFi Experience with streaming, lakehouse, medallion, or enterprise migration initiatives Prior success expanding accounts or managing strategic customers Cloud certifications or advanced ...

data engineering tasks. Experience with cloud platforms such as: AWS, Azure, Google Cloud Platform (GCP) Hands-on experience with data technologies such as: Apache Spark, Databricks, Kafka, Airflow, Snowflake, Hadoop ecosystems Experience designing and maintaining data warehouses and data lakes. Knowledge of DevOps and CI/CD tooling ...

Software Developer Permanent | Hybrid (willing to travel to Newcastle) | Python | Apache | Data | SC cleared Hybrid work arrangement. Office attendance is up to 60%. Location is flexible: London, Leeds, Newcastle, Sheffield, Blackpool, Manchester and Birmingham. You must be willing to travel to Newcastle when required. The Role … looking for Software Developers with strong Python and data processing with Apache Spark responsible to lead the design, development, and operation of data-driven applications and pipelines in a collaborative Dev and DevOps environment. The role focuses on writing and improving application code while working closely with DevOps engineers ...

successful Senior Data Engineer will have the following skills and experience: Strong Python and SQL skills AWS or GCP (NOT Azure) Orchestration, ideally with Airflow or Dagsta but open to others Data warehousing - Snowflake, Redshift, BigQuery (NOT Databricks) Terraform BENEFITS The successful Senior Data Engineer will receive the following ...

successful Senior Data Engineer will have the following skills and experience: Strong Python and SQL skills AWS or GCP (NOT Azure) Orchestration, ideally with Airflow or Dagsta but open to others Data warehousing - Snowflake, Redshift, BigQuery (NOT Databricks) Terraform BENEFITS The successful Senior Data Engineer will receive the following ...

real-time processing. Build and maintain data ingestion frameworks from multiple data sources. Develop scalable data models and data warehouse solutions. Implement workflow orchestration (Airflow/Cloud Composer) including retries, dependencies, and monitoring. Ensure data quality, validation, and governance mechanisms. Design and manage streaming pipelines (Kafka/ ...

Ability to create clean, structured datasets for reporting and analytics Comfortable working in a fast-paced, evolving data environment Nice to Have Experience with Airflow or Dagster Working with JSON/semi-structured data Familiarity with CI/CD pipelines and Git Why Join Work with high-volume, real ...

Glue or EKS Hands on experience with modern analytics stacks such as Snowflake, dbt and Iceberg Familiarity with orchestration tools such as Dagster or Airflow Strong understanding of data modelling concepts and analytics best practice A software engineering mindset, including Git, testing and CI/CD workflows What They ...

Glue or EKS Hands on experience with modern analytics stacks such as Snowflake, dbt and Iceberg Familiarity with orchestration tools such as Dagster or Airflow Strong understanding of data modelling concepts and analytics best practice A software engineering mindset, including Git, testing and CI/CD workflows What They ...

production-grade data pipelines Solid understanding of integration patterns including REST APIs, webhooks, OAuth and event-driven architectures Experience with orchestration tools such as Airflow, Prefect or Dagster Familiarity with cloud environments (AWS, GCP or Azure) Experience with containerisation tools such as Docker and Kubernetes Proven experience integrating multiple ...