26 to 50 of 112 Permanent Apache Airflow Jobs

Building Extract Transform and Load (ETL) pipeline (streaming/batch) using MPP frameworks (Spark, Beam, or other) Experience in orchestrating complex pipelines using Airflow, Dagster or other Proficiency in one or more programming languages: Python, Java, GO or other Experience with AWS or GCP and their products ...

data engineering tasks. Experience with cloud platforms such as: AWS, Azure, Google Cloud Platform (GCP) Hands-on experience with data technologies such as: Apache Spark, Databricks, Kafka, Airflow, Snowflake, Hadoop ecosystems Experience designing and maintaining data warehouses and data lakes. Knowledge of DevOps and CI/CD tooling ...

Experience using Git in a team environment Hands-on experience using AI coding tools (e.g., Copilot, Cursor, Claude, Codex) Nice to have: Experience with Airflow, Dagster or other orchestration tools Experience with data lakes (Synapse, Snowflake) Experience with Docker Experience with streaming technologies (Kafka, RabbitMQ) Experience with Spark ...

Following Distributed computing technologies such as Spark and Databricks Cloud platforms including AWS, Azure, or GCP Workflow orchestration tools such as Airflow CI/CD, version control, and infrastructure as code Desirable Experience within financial services, consulting, or risk environments Degree in Computer Science or a quantitative discipline What ...

databases (Pinecone, Weaviate, FAISS, Milvus) Data & Engineering Excellence Lead the evolution of modern data platforms (Databricks, Snowflake, Synapse, BigQuery) Ensure robust data engineering practices (Airflow, dbt, Prefect) and governance (Collibra, Purview) Drive scalable lakehouse architectures (Delta Lake, Iceberg, Hudi) Cloud, Infrastructure & MLOps Champion multi-cloud engineering across Azure ...

/T-SQL with proven experience refactoring and optimising complex logic Hands-on experience with cloud data warehouses (e.g. BigQuery, Snowflake), ingestion tools (Airflow, Airbyte), and analytics tooling such as dbt and Power BI Strong working knowledge of Git/version control, CI/CD pipelines, and Python ...

Glue or EKS Hands on experience with modern analytics stacks such as Snowflake, dbt and Iceberg Familiarity with orchestration tools such as Dagster or Airflow Strong understanding of data modelling concepts and analytics best practice A software engineering mindset, including Git, testing and CI/CD workflows What They ...

/DS libraries (scikit-learn, numpy, LightlGBM, Pandas, LangChain/LangGraph, TensorFlow, etc..) PySpark AWS cloud infrastructure: EMR, ECS, Athena, etc. MLOps: Terraform, Docker, Airflow, MLFlow More information: Enjoy fantastic perks like private healthcare & dental insurance, a generous work from abroad policy, 2-for-1 share purchase plans ...

Redux, Redux Hooks, and other modern state management libraries. Kotlin on Android and Swift on iOS. Data Analytics/Engineering: SQL, Python, Snowflake, dbt, Airflow & Looker. AI at Marshmallow: Sierra.ai for conversational AI, Python and LangChain for agentic AI solutions, SageMaker and MLflow for Machine Learning, Cursor and Claude ...

data pipelines, managing large datasets, and enabling engineering and AI teams to deliver high-performance data-driven products. Key Technical Requirements • Python • SQL • Spark • Airflow • Data pipelines • AWS or GCP Experience Required • 4+ years in data engineering • Experience working with large-scale structured and unstructured datasets • Strong ETL development ...

real-time processing. Build and maintain data ingestion frameworks from multiple data sources. Develop scalable data models and data warehouse solutions. Implement workflow orchestration (Airflow/Cloud Composer) including retries, dependencies, and monitoring. Ensure data quality, validation, and governance mechanisms. Design and manage streaming pipelines (Kafka/ ...

technical solutions meet business needs. Experience with data ingestion tools, like Fivetran. Advantageous Exposure to deploying applications with Kubernetes. Experience with Data Orchestrator tools (Airflow, Prefect, etc.). Experience with Data Observability tools (Montecarlo, Great Expectations, etc.). Experience with Data Catalog tools (Amundsen, OpenMetadata, etc.). Benefits Winning ...

Experience with embeddings, vector databases, RAG, LLM-based enrichment, or knowledge graph completion. Experience with MLOps tools such as Docker, Kubernetes, MLflow, Weights & Biases, Airflow, or similar. Experience with AWS, GCP, or Azure. Experience in construction, architecture, BIM, building materials, technical product data, catalogues, standards, or compliance-heavy data ...

production-grade data pipelines Solid understanding of integration patterns including REST APIs, webhooks, OAuth and event-driven architectures Experience with orchestration tools such as Airflow, Prefect or Dagster Familiarity with cloud environments (AWS, GCP or Azure) Experience with containerisation tools such as Docker and Kubernetes Proven experience integrating multiple ...

. ML workflow orchestration and serving: Operate the systems researchers use to launch experiments, data jobs, and production endpoints — workflow engines (e.g., Kubeflow Pipelines, Airflow), GPU schedulers (e.g., Volcano, Slurm), experiment trackers (e.g., MLflow, Weights & Biases), and managed‐inference platforms (e.g., Modal, Triton) — and maintain a launcher ...

Azure, or Google Cloud) including data lakes, warehouses, and lakehouse solutions. Transform and process structured and unstructured data using modern ETL/ELT frameworks (Apache Spark, Airflow, dbt). Collaborate closely with product managers, analysts, and software developers to ensure seamless integration and high-quality data availability. Develop ...

upholding data quality and governance standards What we're looking for Strong SQL and Python skills, with hands-on experience in dbt, Databricks, Airflow or Looker Solid analytics engineering fundamentals — data modelling, warehousing, performance optimisation, observability and governance Experience in a marketplace or e-commerce environment, with comfort around ...

upholding data quality and governance standards What we're looking for Strong SQL and Python skills, with hands-on experience in dbt, Databricks, Airflow or Looker Solid analytics engineering fundamentals — data modelling, warehousing, performance optimisation, observability and governance Experience in a marketplace or e-commerce environment, with comfort around ...

dimensional modelling Experience working with Snowflake or similar cloud data warehouses such as BigQuery Good Python capability, including orchestration tools such as Airflow Proven experience building and managing data ingestion pipelines Cloud experience within AWS or GCP environments Exposure to infrastructure as code tools such as Terraform Confidence leading ...

dimensional modelling Experience working with Snowflake or similar cloud data warehouses such as BigQuery Good Python capability, including orchestration tools such as Airflow Proven experience building and managing data ingestion pipelines Cloud experience within AWS or GCP environments Exposure to infrastructure as code tools such as Terraform Confidence leading ...

Deep experience with Spark and/or Databricks Strong cloud expertise across AWS, Azure and/or GCP Experience with orchestration technologies such as Airflow A track record of delivering scalable, resilient and high-quality data products The ability to influence customers and senior stakeholders Experience developing engineering capabilities ...

orchestration It would be nice if you could bring (desirables): Professional experience with PySpark, MySQL, Docker, Kubernetes Experience with best-practice pipeline orchestration software (Airflow, Dagster). Experience migrating code from prototype notebooks to .py, while maintaining logic and transparency of underlying code Owned a project across a whole ...

infrastructure. Analyses and reports on test activities, results, issues and risks. Production data analysis to identify root cause of production environment issues. Using Apache Airflow for task orchestration and reviewing output logs. Basic understanding of EMR and reviewing output logs. Basic understanding of data processing using Apache ...

complete features, though it's not the primary focus of the role. Our Stack (100% Open Source) Backend: Golang, Python Infrastructure & Orchestration: Docker, Kubernetes, Airflow, Salt, Ansible, Chef Data: PostgreSQL, Redis, Kafka API: REST, GraphQL Frontend: React with TypeScript What you’ll do Own backend systems — design, build … networking, filesystem concepts Configuration management experience with Salt, Ansible, Chef, or similar tools Container and orchestration expertise — Docker, Kubernetes, or related technologies Workflow orchestration — Apache Airflow or similar pipeline frameworks Database design skills — schema design and query optimization in PostgreSQL or similar RDBMS Distributed systems knowledge — you think ...

technical trade‐offs, backed by a previous background as an experienced software engineer. Experience with data platform and ETL/orchestration technologies such as Apache Airflow, Spark, AWS Glue, and Snowflake. Experience with or strong advocacy for AI developer tooling such as Claude Code, Copilot, Windsurf, and Gemini. ...