126 to 150 of 308 Remote/Hybrid Apache Airflow Jobs

secureCost-conscious: optimised warehouse spend, storage strategies, or job efficiency, and treat compute as a resource to manage, not ignorePlatform fluency: worked across orchestration (Airflow, Dagster), warehouses (Snowflake, BigQuery, Databricks), ingestion (Fivetran, Airbyte, custom), and transformation (dbt, Spark)AI Native: experienced in agentic coding workflows (context, harness, loop ...

functionally with Product/Design/Engineering and bring teams along on decisions. To share extra context, our current tech stack includes GCP, BigQuery, Airflow, Python, SQL on the Data side and AWS, Kotlin, Javascript, Typescript on the Product side while our infrastructure is containerised with Kubernetes. Experience across ...

Description Job DescriptionThe Software Engineer shall be responsible for creating and maintaining data workflows and automation pipelines using Apache Airflow. This role focuses on building reliable, scalable, and observable workflow orchestration solutions that support data engineering, analytics, and operational use cases. The engineer will collaborate closely with data engineers … Linux CLI and Linux tools Experience developing Bash scripts to automate manual processes Recent software development experience using Python and Java Experience using Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows Experience using Distributed Big Data processing engines including Apache Spark ...

Description Job DescriptionThe Software Engineer shall be responsible for creating and maintaining data workflows and automation pipelines using Apache Airflow. This role focuses on building reliable, scalable, and observable workflow orchestration solutions that support data engineering, analytics, and operational use cases. The engineer will collaborate closely with data engineers … Linux CLI and Linux tools Experience developing Bash scripts to automate manual processes Recent software development experience using Python and Java Experience using Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows Experience using Distributed Big Data processing engines including Apache Spark ...

while mentoring engineers and setting engineering standards. You'll work closely with stakeholders to translate requirements into robust data solutions, using Databricks, Snowflake, dbt, Airflow, Python and SQL in production environments. #J-18808-Ljbffr ...

equivalent. Experience with big data sets and big data warehouses Experience with proxies, anti-bot measures and scraper monitoring. Experience with Dagster, Airflow, Prefect or another data orchestrator. Comfort working with TypeScript/Node.js. A pragmatic, hands-on approach and strong problem-solving skills. Desirable: Geospatial data experience (PostGIS ...

Terraform, Bicep or Pulumi) Implementing monitoring and observability for production systems, including metrics, alerting, logging, and dashboarding (e.g. Prometheus, Grafana) Pipeline orchestration using Dagster, Airflow, Prefect, or similar Bonus points if you have: Practical experience with model serving infrastructure – batch and/or real‐time inference at scale. Experience ...

track record of delivering impact Desirable: Experience with GenAI/LLM evaluation, AI safety, prompt engineering, experimentation frameworks or data pipeline technologies such as Airflow or Dagster. What we’ll provide you This is a unique role that will put you in the position to be part ...

monitoring infrastructure that catches drift and quality degradation before business KPIs move. You'll work on a GCP-native stack — Vertex AI, BigQuery, dbt, Airflow, and Terraform — on problems that matter from day one, with the autonomy to shape how the function scales and who joins it next. ...

unstructured data sources including Cassandra HBase Kafka and Spark* Strong ETL and data pipeline experience particularly with modern big data toolsets such as Airflow and Nifi or equivalent* Strong programming background in either Java Python or Scala* Experience in modern engineering practices and technologies* Good working knowledge of data ...

multi-TB or PB scale Strong software engineering skills Experience delivering & maintaining production systems Experience with workflow orchestration for parallelised jobs; Flyte preferred, but Airflow, Prefect or similar tools are also relevant Experience building reliable, observable data systems, including retry strategies, backfills, data integrity checks, lifecycle/state tracking ...

organisation* Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI* Expertise in Streaming Data, Event Driven Design, Data Governance ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

organisation Deep understanding & hands-on leverage of Modern, Cloud scale Data Technologies, with specific expertise in Distributed Data Processing architectures (Spark, PySpark, Python, Airflow), ELT data processing underpinning Data LakeHouses and Data Products for scaled use within ML & AI Expertise in Streaming Data, Event Driven Design, Data Governance ...

cloud‐native data architecture (AWS/GCP/Snowflake). Architect and implement scalable ingestion and transformation pipelines using Fivetran, Databricks, and Airflow/DBT. Design and code modular data services and APIs that unify campaign, conversion, and audience datasets across 20+ marketing platforms. Build and optimize schemas, transformations … Hands‐on experience as a Data Engineer or Data Platform Developer, including direct contributions to production codebases. Expert‐level proficiency in Python, SQL, and Apache Spark for data pipeline development. Proven experience building ELT workflows using Fivetran, Airflow, DBT, or Databricks. Solid understanding of data modeling, dimensional design ...

platformsAdvanced Python and SQL skills, with experience in modern engineering practices such as CI/CD and TDDExperience with big data frameworks such as Apache Spark or similar technologiesSolid understanding of data modelling, data warehousing, OLTP and data lake architecturesExperience working with cloud ecosystems such as AWS, Azure … GCPFamiliarity with tools such as Kafka, Airflow, Hive, Delta, or similar technologiesStrong understanding of software engineering principles and production supportExperience with Linux, containerisation and enterprise scheduling toolsKnowledge of data governance, security and data privacy principles including GDPRConfident communicator, able to present complex technical concepts to varied audiencesA proactive, self ...

Kubernetes - e.g. RDMA, DPUs, Infiniband, many-core CPUs Experience with declarative CI/CD tools such as ArgoCD Experience with workflow engines such as Apache Airflow or Argo Workflows Experience with infrastructure automation Cloud engineering experience with at least one cloud service provider Experience ...

ideal candidate will have strong experience in a lead data engineering position, extensive GCP know-how (BigQuery, Dataflow, Pub/Sub, Cloud Storage, Airflow), and proficiency in Python and SQL. A focus on scalable data pipelines and CI/CD is essential. #J-18808-Ljbffr ...

focused on what works in production. Comfortable owning ambiguity and making trade-overs. Languages: English A plus, but not required: workflow orchestration tools (Temporal, Airflow, Step Functions); experience building internal tools or agents; familiarity with regulated or enterprise environments; exposure to MLOps or AI evaluation frameworks. Why Planet Planet ...

including web scraping, parsing, proxy management, and structured feed ingestion. Pipeline Architect: Hands-on experience with batch-style data workflows using tools like dbt, Airflow, or similar equivalents. Cloud Native: Deep expertise in AWS infrastructure (RDS, ECS, S3, Lambda, OpenSearch). Observability Minded: Familiar with modern stacks for performance ...

including web scraping, parsing, proxy management, and structured feed ingestion. Pipeline Architect: Hands-on experience with batch-style data workflows using tools like dbt, Airflow, or similar equivalents. Cloud Native: Deep expertise in AWS infrastructure (RDS, ECS, S3, Lambda, OpenSearch). Observability Minded: Familiar with modern stacks for performance ...

including web scraping, parsing, proxy management, and structured feed ingestion. Pipeline Architect: Hands-on experience with batch-style data workflows using tools like dbt, Airflow, or similar equivalents. Cloud Native: Deep expertise in AWS infrastructure (RDS, ECS, S3, Lambda, OpenSearch). Observability Minded: Familiar with modern stacks for performance ...

including web scraping, parsing, proxy management, and structured feed ingestion. Pipeline Architect: Hands-on experience with batch-style data workflows using tools like dbt, Airflow, or similar equivalents. Cloud Native: Deep expertise in AWS infrastructure (RDS, ECS, S3, Lambda, OpenSearch). Observability Minded: Familiar with modern stacks for performance ...

including web scraping, parsing, proxy management, and structured feed ingestion. Pipeline Architect: Hands-on experience with batch-style data workflows using tools like dbt, Airflow, or similar equivalents. Cloud Native: Deep expertise in AWS infrastructure (RDS, ECS, S3, Lambda, OpenSearch). Observability Minded: Familiar with modern stacks for performance ...

including web scraping, parsing, proxy management, and structured feed ingestion. Pipeline Architect: Hands-on experience with batch-style data workflows using tools like dbt, Airflow, or similar equivalents. Cloud Native: Deep expertise in AWS infrastructure (RDS, ECS, S3, Lambda, OpenSearch). Observability Minded: Familiar with modern stacks for performance ...