651 to 675 of 945 Apache Airflow Jobs

platforms - execution roles, service roles, cross-account access, Lake Formation, SSO/SAML MLflow or equivalent experiment tracking SageMaker Pipelines or similar workflow orchestration (Airflow, Step Functions) Model serving - real-time endpoints, batch transform, auto-scaling, endpoint monitoring Snowflake as a data source for ML pipelines Kubernetes ...

ideally within a large‐scale or regulated environment. AWS Experience: Strong working knowledge of AWS and cloud‐first data pipeline orchestration tools such as Airflow, Glue, or EMR. A Security Mindset: Understanding of compliance and security when managing sensitive data pipelines. Strong Automation Skills: Comfortable using Python and Shell ...

platforms - execution roles, service roles, cross-account access, Lake Formation, SSO/SAML MLflow or equivalent experiment tracking SageMaker Pipelines or similar workflow orchestration (Airflow, Step Functions) Model serving - real-time endpoints, batch transform, auto-scaling, endpoint monitoring Snowflake as a data source for ML pipelines Kubernetes ...

narratives Entrepreneurial mentality: You can take work beyond the analysis and drive outcomes. Useful additional experience: Experience in Fintech domain Knowledge of utilities like Airflow/Github/DBT Experience with user-facing products and data Please note: we are not able to offer visa sponsorship for this role ...

labeling workflows, ontologies, consensus systems, QA, or label-store integrations Experience with Go, Rust, or TypeScript for services Experience with orchestration tools such as Airflow or Prefect Experience with metadata and lineage tools such as MLflow or Weights & Biases Exposure to edge or sensor data, including audio, sonar, video ...

labeling workflows, ontologies, consensus systems, QA, or label-store integrations Experience with Go, Rust, or TypeScript for services Experience with orchestration tools such as Airflow or Prefect Experience with metadata and lineage tools such as MLflow or Weights & Biases Exposure to edge or sensor data, including audio, sonar, video ...

large‐scale or regulated environment. AWS Experience: You have a strong working knowledge of AWS and cloud‐first data pipeline orchestration tools (like Airflow, Glue, or EMR). A Security Mindset: You understand the importance of compliance and security when managing sensitive data pipelines. Strong Automation Skills ...

will include Design, build and maintain robust, scalable data pipelines that support data science workflows and advanced analytics use cases Develop and manage complex Airflow orchestration, ensuring pipelines are reliable, observable and maintainable Work hands-on with Databricks suite to deliver high-quality data engineering solutions for data science ...

what's at stake for real people, and that's what gives our intensity its purpose. Technologies we use Claude Code, Kubernetes, AWS, Airflow, dlt, dbt, Python, TypeScript, React, Go Please note: we are unable to sponsor or take over sponsorship of an employment visa at this time. Compensation ...

what's at stake for real people, and that's what gives our intensity its purpose. Technologies we use Claude Code, Kubernetes, AWS, Airflow, dlt, dbt, Python, TypeScript, React, Go Please note: we are unable to sponsor or take over sponsorship of an employment visa at this time. Compensation ...

large-scale or regulated environment.* AWS Experience: You have a strong working knowledge of AWS and cloud-first data pipeline orchestration tools (like Airflow, Glue, or EMR).* A Security Mindset: You understand the importance of compliance and security when managing sensitive data pipelines.* Strong Automation Skills ...

development, dashboard creation, or data ingestion pipelines. Containerization & Orchestration: Hands-on experience with Kubernetes or Rancher architectures. Workflow Automation: Experience orchestrating data pipelines using Apache Airflow. The Benefits Package Wyetech believes in generously supporting employees as they prepare for retirement. The company automatically contributes 20% of each employee ...

Expert‐level Python development skills with production experience at scale. Strong Kubernetes experience with production workloads. Hands‐on experience with workflow orchestration tools (Temporal, Airflow, Kubeflow, or similar). Cloud platform expertise (AWS required, Azure & GCP beneficial). Experience with data processing frameworks (Spark, Athena, Ray, or similar). ...

idiomatic Python used for reliable, well-tested systems (not exploratory notebooks). Design and operation of large-scale ETL/ELT pipelines, including orchestration (Airflow, Prefect, or similar) and reliability patterns such as retries, backfills, and exactly-once semantics. Deep experience with Azure, including AKS, Helm, EventHub, observability tools … Advanced experience operating Delta tables at scale, covering transaction semantics, schema evolution, partitioning, and compaction, alongside highly optimised analytical query execution using DuckDB or Apache Spark. Production-grade relational database expertise, particularly PostgreSQL (query tuning, indexing, partitioning, replication, and schema design across OLTP and analytics). Strong KDB experience ...

data engineering Experience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal) Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governance Practical cloud-native experience (AWS, Azure, or GCP) Ability … hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with emerging technologies in data engineering ...

applications using TypeScript, HTML, and CSS Experience with JavaScript frameworks such as React, Angular, and Vue Familiar with Distributed Big Data processing engines including Apache Spark Experience using Jupyter Notebook Experience with data wrangling and preprocessing using tools such as pandas, NumPy Experience working with structured, semi-structured … Parquet, JSON, CSV, XML Familiarity with data quality concepts, data validation, and anomaly detection Experience with Git Source Control System. Required Skills Experience using Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows Experience with SQL technologies such as MySQL, MariaDB, and PostgreSQL ...

applications using TypeScript, HTML, and CSS Experience with JavaScript frameworks such as React, Angular, and Vue Familiar with Distributed Big Data processing engines including Apache Spark Experience using Jupyter Notebook Experience with data wrangling and preprocessing using tools such as pandas, NumPy Experience working with structured, semi-structured … Parquet, JSON, CSV, XML Familiarity with data quality concepts, data validation, and anomaly detection Experience with Git Source Control System. Required Skills Experience using Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows Experience with SQL technologies such as MySQL, MariaDB, and PostgreSQL ...

PostgreSQL (or similar relational databases) and NoSQL databases. Experience with vector databases such as Qdrant, Milvus or pgvector . Familiarity with technologies such as Apache Spark, Airflow, Kafka, Elasticsearch/OpenSearch, Pandas and Polars . Strong understanding of data modelling, storage architecture and performance optimisation. Desirable Experience working ...

Senior Data Engineer to design, build, and operate enterprise-scale data solutions. You will leverage Snowflake with hands-on Python, PySpark, AWS and Airflow to create scalable data platforms and robust analytics capabilities. The role emphasizes technical leadership, governance, and collaboration with cross-functional teams to deliver high-performance ...

devices and platforms to surface key metrics. The role emphasizes coding in Python, strong SQL, and experience with large data ecosystems (HDFS, Hive, Spark, Airflow). You will influence architecture and own initiatives from concept to delivery. Hybrid work from Cambridge Office. #J-18808-Ljbffr ...

management * Software testing lifecycle/defect management * CI/CD & DevOps practices Desirable: HMRC/public sector experience, AWS/Azure/SAS PaaS, Airflow or SAS Migration Factory experience. If you are interested in this role or wish to apply, please submit your CV. ...

technical direction and influencing engineering practices across teams; Hands-on experience with cloud ML platforms such as AWS SageMaker and orchestration tools such as Airflow and Step Functions; Strong communication skills and ability to collaborate across Data Science, Engineering, and Data Platforms; Comfortable operating at the boundary between experimentation ...

database platforms (e.g., PostgreSQL, ClickHouse, MSSQL, Redis, FoundationDB).Familiarity with specific middleware (e.g., Kafka, Consul), HPC schedulers (e.g., Slurm), Kubernetes, or workflow orchestrators (e.g., Airflow, Prefect).Experience with on-premise deployments of development tools like GitLab, Artifactory, CICD, or JupyterHub. Knowledge of other programming languages such as Golang, Rust ...

classification taxonomies. Working with model-derived inputs, understanding that NLP-generated exposures carry measurement error and revise over time. Task orchestration (Dagster or Airflow) and S3-based data flows. AWS fluency and CI/CD discipline. Competitive salary plus EMI share options 25 days holiday + bank holidays Private ...

Experience with ERP systems, especially SAP (maintenance workflows, master data, integrations) Operations research or constraint programming background Data transformation and orchestration tools (dbt, Prefect, Airflow) B2B SaaS Why Join Ayama: Exclusive Ecosystem: Unparalleled access to AI Fund's experts and Dr. Andrew Ng's network of AI luminaries. Real ...