151 to 175 of 928 Permanent Apache Airflow Jobs

Configuration Management (CM) tools and Web Services implementation. The Software Engineer shall be responsible for creating and maintaining data workflows and automation pipelines using Apache Airflow. This role focuses on building reliable, scalable, and observable workflow orchestration solutions that support data engineering, analytics, and operational use cases. The engineer … Linux CLI and Linux tools Experience developing Bash scripts to automate manual processes Recent software development experience using Python and Java Experience using Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows Experience using Distributed Big Data processing engines including Apache Spark ...

Configuration Management (CM) tools and Web Services implementation. The Software Engineer shall be responsible for creating and maintaining data workflows and automation pipelines using Apache Airflow. This role focuses on building reliable, scalable, and observable workflow orchestration solutions that support data engineering, analytics, and operational use cases. The engineer … Linux CLI and Linux tools Experience developing Bash scripts to automate manual processes Recent software development experience using Python and Java Experience using Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows Experience using Distributed Big Data processing engines including Apache Spark ...

tooling, automated testing and reliable deployment processes. Startup Mindset: Proactive, resourceful and comfortable taking responsibility in a small, fast‐moving engineering team. Experience with Apache Airflow, Kubernetes, real‐time data platforms, AI/ML infrastructure, LLM agents, GPU infrastructure or financial‐market systems would be useful ...

code Requirements: Must have strong, commercial AWS (contract) Data Engineering experience Must bring solid knowledge of data warehousing and modelling Experience with Airflow and infrastructure as a code (Terraform, CloudFormation) would be beneficial Must have experience with Redshift Nice to have: Experience with dbt/lake formation ...

tools. Experience building and optimising scalable data pipelines and data warehouses. Knowledge of AWS technologies such as Redshift, S3 and Athena. Experience with Airflow, Fivetran or similar orchestration and integration tools. Strong Linux/Unix and Bash scripting skills. A solid understanding of data quality, governance, lineage and monitoring. ...

Arrow, plus an analytical engine, e.g. DuckDB, or a warehouse such as Snowflake. Orchestrated batch pipelines you've operated, not just written: Dagster or Airflow, S3-based data flows, and a habit of testing outputs for correctness rather than only for exceptions. Applied ML in production: embeddings and semantic ...

database querying languages Experience in large‐scale data processing, microservices, API design, Kafka, Redis, MemCached, Observability tools (Dynatrace, Splunk, Grafana), and Orchestration tools (Airflow, Temporal) Demonstrated experience leading effective use of enterprise‐authorized AI‐assist­ed software development tools within the work environment (e.g., for coding, code review, test acceleration ...

systems, inventory extraction from application metadata tables (reporting opening). Columnar and lake formats (Parquet or similar), large object stores, orchestration platforms (Airflow or similar). Working-level knowledge of graph databases; data contracts and data quality frameworks; catalogue platforms (DataHub or similar) on the ingestion side. ...

data modelling Databricks as our datalake platform Kubernetes for data services Orchestra for task orchestration Terraform for infrastructure Lightdash for data applications Airflow and Astronomer for job scheduling and tracking Circle CI for continuous deployment Parquet and Delta file formats on S3 for data lake storage Spark for data ...

influence technical and non‐technical stakeholders. Desirable Infrastructure‐as‐code (Terraform), Kubernetes, or Go/TypeScript experience. Experience in regulated industries. Familiarity with dbt, Airflow and event‐driven architectures. Admiral: Where You Can We take pride in being a diverse and inclusive business. It's a place where ...

experience building ML infrastructure, including model serving, training pipelines, feature stores, embeddings, or ML observability. Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines. Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization. ...

experience building ML infrastructure, including model serving, training pipelines, feature stores, embeddings, or ML observability. Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines. Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization. ...

experience building ML infrastructure, including model serving, training pipelines, feature stores, embeddings, or ML observability. Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines. Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization. ...

experience building ML infrastructure, including model serving, training pipelines, feature stores, embeddings, or ML observability. Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines. Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization. ...

experience building ML infrastructure, including model serving, training pipelines, feature stores, embeddings, or ML observability. Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines. Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization. ...

experience building ML infrastructure, including model serving, training pipelines, feature stores, embeddings, or ML observability. Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines. Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization. ...

experience building ML infrastructure, including model serving, training pipelines, feature stores, embeddings, or ML observability. Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines. Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization. ...

experience building ML infrastructure, including model serving, training pipelines, feature stores, embeddings, or ML observability. Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines. Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization. ...

experience building ML infrastructure, including model serving, training pipelines, feature stores, embeddings, or ML observability. Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines. Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization. ...

experience building ML infrastructure, including model serving, training pipelines, feature stores, embeddings, or ML observability. Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines. Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization. ...

experience building ML infrastructure, including model serving, training pipelines, feature stores, embeddings, or ML observability. Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines. Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization. ...

experience building ML infrastructure, including model serving, training pipelines, feature stores, embeddings, or ML observability. Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines. Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization. ...

experience building ML infrastructure, including model serving, training pipelines, feature stores, embeddings, or ML observability. Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines. Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization. ...

experience building ML infrastructure, including model serving, training pipelines, feature stores, embeddings, or ML observability. Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines. Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization. ...

experience building ML infrastructure, including model serving, training pipelines, feature stores, embeddings, or ML observability. Experience with modern data platforms such as BigQuery, Airflow, Spark, Beam/Dataflow, or streaming pipelines. Practical experience building production systems with LLMs or generative AI, including orchestration, provider APIs, observability, and performance optimization. ...