1 to 25 of 59 Prefect Jobs

/DatabricksAWS Glue/Redshift/EMRGoogle BigQuery/DataflowEnsure systems are secure, scalable, and cost-efficientData Pipeline & Workflow ManagementDevelop orchestration workflows (e.g., Airflow, Prefect, Azure Data Factory)Monitor and troubleshoot pipelines to ensure high availability and performanceImplement robust error handling, logging, and alertingData Quality & GovernanceEnsure data quality, integrity ...

highly advantageous, particularly: Azure Data Factory Azure SQL Azure Data Lake/ADLS Azure DevOps Microsoft Fabric Azure Databricks Experience with PySpark, Airflow, Dagster, Prefect or other orchestration technologies would also be useful. We're particularly interested in people who have worked with modern data platforms and understand the principles ...

client-ready deliverables.Desirable Experience with advanced analytical techniques, such as machine learning, predictive modelling, or optimisation.Familiarity with data orchestration tools (e.g., Airflow, dbt, Prefect, Cloud Composer).Experience with containerisation or infrastructure-as-code (e.g., Docker, Kubernetes, Terraform).Knowledge of data warehousing concepts and technologies.Experience contributing to proposals, scoping documents ...

client-ready deliverables.Desirable Experience with advanced analytical techniques, such as machine learning, predictive modelling, or optimisation.Familiarity with data orchestration tools (e.g., Airflow, dbt, Prefect, Cloud Composer).Experience with containerisation or infrastructure-as-code (e.g., Docker, Kubernetes, Terraform).Knowledge of data warehousing concepts and technologies.Experience contributing to proposals, scoping documents ...

client-ready deliverables.Desirable Experience with advanced analytical techniques, such as machine learning, predictive modelling, or optimisation.Familiarity with data orchestration tools (e.g., Airflow, dbt, Prefect, Cloud Composer).Experience with containerisation or infrastructure-as-code (e.g., Docker, Kubernetes, Terraform).Knowledge of data warehousing concepts and technologies.Experience contributing to proposals, scoping documents ...

client-ready deliverables.Desirable Experience with advanced analytical techniques, such as machine learning, predictive modelling, or optimisation.Familiarity with data orchestration tools (e.g., Airflow, dbt, Prefect, Cloud Composer).Experience with containerisation or infrastructure-as-code (e.g., Docker, Kubernetes, Terraform).Knowledge of data warehousing concepts and technologies.Experience contributing to proposals, scoping documents ...

client-ready deliverables.Desirable Experience with advanced analytical techniques, such as machine learning, predictive modelling, or optimisation.Familiarity with data orchestration tools (e.g., Airflow, dbt, Prefect, Cloud Composer).Experience with containerisation or infrastructure-as-code (e.g., Docker, Kubernetes, Terraform).Knowledge of data warehousing concepts and technologies.Experience contributing to proposals, scoping documents ...

managed data services.Strong experience with modern data warehousing/lakehouse tools (e.g., Snowflake, BigQuery, Redshift, Databricks).Hands-on orchestration experience (e.g., Airflow, Dagster, Prefect) and CI/CD for data pipelines.Experience with streaming and event-driven data (e.g., Kafka, Kinesis, Pub/Sub) is highly desirable.Proficiency with infrastructure as code ...

data quality practice. A collaborative mindset and a genuine interest in building things well. Desirable Experience with orchestration tools such as Airflow, Dagster, Prefect or Azure Data Factory. Experience working within AWS, Azure, GCP or another enterprise cloud ecosystem. Experience with complex customer, transactional, digital or behavioural data. Exposure ...

columnar data formats like Parquet. MLOps Exposure: Experience with tools and frameworks for training, tracking, deploying, and monitoring models in production (e.g., MLflow, Optuna, Prefect, or Airflow). Background in supply chain forecasting, enterprise AI, or retail analytics M.S. or Ph.D. in Computer Science, Artificial Intelligence, Statistics, Data Science ...

columnar data formats like Parquet. MLOps Exposure: Experience with tools and frameworks for training, tracking, deploying, and monitoring models in production (e.g., MLflow, Optuna, Prefect, or Airflow). Desirable Background in supply chain forecasting, enterprise AI, or retail analyticsM.S. or Ph.D. in Computer Science, Artificial Intelligence, Statistics, Data Science ...

/CD experience. Highly Desirable Familiarity with OneTick, PostgreSQL, ClickHouse, Snowflake, Databricks, kdb+ or similar. Order book reconstruction Tick-to-bar aggregation Airflow, Dagster, Prefect or MLflow Experience supporting quantitative research or trading systems Multi-asset market data (Equities, Futures, FX, Options, ETFs, CFDs) What We're Looking ...

data transformation tools (such as dbt). Pipeline mastery: Practical experience building and operating batch or streaming pipelines using orchestration tools like Airflow, Prefect, or Dagster. Strong communicator: Skilled at writing clear technical specifications and explaining complex trade-offs to both technical and non-technical colleagues. Growth mindset: Open ...

data pipelines and distributed data platforms Knowledge of Kafka, Spark and modern data processing technologies Experience with orchestration tools such as Airflow, Dagster or Prefect Hands-on experience with Docker, Kubernetes and containerised deployments CI/CD experience with a focus on production ownership and reliability Understanding of data quality ...

native platforms in AWS and/or GCP.Experience designing and supporting data pipelines within complex technical environments.Familiarity with workflow orchestration tools such as Airflow, Prefect or Dagster.Experience implementing monitoring, logging, observability and platform governance practices.Strong understanding of software engineering principles, testing and deployment best practices.Ability to work collaboratively with technical ...

platforms in AWS and/or GCP.Experience designing and supporting data pipelines within complex technical environments. Familiarity with workflow orchestration tools such as Airflow, Prefect or Dagster. Experience implementing monitoring, logging, observability and platform governance practices. Strong understanding of software engineering principles, testing and deployment best practices. Ability to work ...

equivalent practical experience. Preferred Skills Experience with gradient-boosted tree models, neural networks, and optimization solvers in production. Familiarity with data orchestration tools (e.g., Prefect, Airflow, dbt) and modern data lake architectures. Experience working closely with data scientists to operationalize research code. Multi-cloud experience across AWS, GCP, and Azure. ...

surrogate modeling, Bayesian optimization). Hands-on work with multi-agent systems, knowledge graphs (Neo4j), or agent frameworks/SDKs. Familiarity with Airflow or Prefect, PyTorch, and related ML/LLM tooling. Direct experience with commercial LIMS/ELN or analytical platforms such as Genedata, CDD Vault, Virscidian Analytical Studio ...

surrogate modeling, Bayesian optimization). Hands-on work with multi-agent systems, knowledge graphs (Neo4j), or agent frameworks/SDKs. Familiarity with Airflow or Prefect, PyTorch, and related ML/LLM tooling. Direct experience with commercial LIMS/ELN or analytical platforms such as Genedata, CDD Vault, Virscidian Analytical Studio ...

knowledge of Kubernetes/EKS, comfortable deploying to and reasoning about containerised workloads in a managed K8s environment. Experience orchestrating pipelines with Dagster, Airflow, Prefect, or similar. Solid grasp of ETL concepts and production‐grade pipeline design. Beneficial Skills Interest in chemistry, life sciences, robotics, or manufacturing. Hands‐on experience ...

Kafka), Linux environments, and at least one major cloud provider (AWS, GCP, Azure). Familiarity with data pipeline and workflow management tools (e.g., Airflow, Prefect, or Step Functions). Strong software engineering fundamentals: unit testing, CI/CD, code reviews, and design documentation. Additional Information Standard Hours/Schedule ...

equivalent. Experience with big data sets and big data warehouses Experience with proxies, anti-bot measures and scraper monitoring. Experience with Dagster, Airflow, Prefect or another data orchestrator. Comfort working with TypeScript/Node.js. A pragmatic, hands-on approach and strong problem-solving skills. Desirable: Geospatial data experience (PostGIS, GeoPandas ...

Python used for reliable, well-tested systems (not exploratory notebooks).• Design and operation of large-scale ETL/ELT pipelines, including orchestration (Airflow, Prefect, or similar) and reliability patterns such as retries, backfills, and exactly-once semantics.• Deep experience with Azure, including AKS, Helm, EventHub, observability tools (e.g. ...

Machine Learning systems into production.Desirable experienceNVIDIA Jetson or other edge AI platforms.NVIDIA DeepStream, GStreamer or FFmpeg.Kubernetes, Kubeflow, ECS, Nomad or other container-orchestration platforms.Airflow, Prefect, Dagster, Metaflow or similar workflow tools.Kafka, Pulsar, Redis Streams, RabbitMQ or other event-driven technologies.Triton Inference Server, Ray Serve, TorchServe, BentoML or Seldon Core.DVC, LakeFS ...

compute, and storage optimization on AWS or GCP. Strong candidates may also have experience with Data pipelining for AI/ML workflows using Airflow, Prefect, or Dagster. Build system expertise with Bazel, CMake, or distributed build systems. Secrets management tools such as Vault, SOPS, AWS Secrets Manager, or GCP Secret ...