176 to 200 of 887 Apache Airflow Jobs

delivery (CI/CD) pipelines, containerisation, and workflow orchestration. Familiar with ETL/ELT frameworks, and experienced with Big Data Processing Tools (e.g. Spark, Airflow, Hive, etc.) Knowledge of programming languages (e.g. Java, Python, SQL) Hands-on experience with SQL/NoSQL database design. Degree in STEM, or similar ...

Experience with MLOps tooling and workflow orchestration - experiment tracking and model registry (MLflow or equivalent), and pipeline orchestration on Kubernetes (Kubeflow, Argo Workflows, or Airflow) - supporting automated retraining, model versioning, and drift detection Expert experience developing ML algorithms - supervised/unsupervised learning, neural networks, and NLP - with hands ...

metrics layers. Experience applying AI to Analytics Engineering workflows or developing AI-ready data products. Knowledge of orchestration and ingestion tools such as Airflow,DagsterorFivetran. Python experience for automation or data engineering tasks. Experience working in regulated financial services or another highly data-driven organisation. Snowflake experience, given ...

implementation of modern data architectures and concepts such as cloud services (e.g., AWS, Azure, GCP), real-time data distribution (e.g., Kafka, Kinesis, DataFlow, Airflow), NoSQL (e.g., MongoDB, DynamoDB, HBase, CosmosDB), GraphQL and modern data warehouse tools including Snowflake and DataBricksAbility to think strategically and relate architectural decisions and recommendations ...

implementation of modern data architectures and concepts such as cloud services (e.g., AWS, Azure, GCP), real-time data distribution (e.g., Kafka, Kinesis, DataFlow, Airflow), NoSQL (e.g., MongoDB, DynamoDB, HBase, CosmosDB), GraphQL and modern data warehouse tools including Snowflake and DataBricksAbility to think strategically and relate architectural decisions and recommendations ...

substitute. Strong platform engineering foundations: cloud (Azure and/or AWS), containers, infrastructure as code, CI/CD, and workflow orchestration (Airflow, Databricks Workflows or similar). Hands-on with ML lifecycle tooling - experiment tracking, model registry (MLflow or equivalent), and CI/CD for models rather than only ...

Experience with Infrastructure as Code (Terraform or CloudFormation). Automation/Al: Proficiency in Python scripting for automation. Experience with workflow orchestration tools (e.g., Airflow, Step Functions). Basic experience integrating Al/ML models into applications. DevOps: Basic understanding of Cl/CD pipelines, containerization (Docker), and cloud ...

Working knowledge of Kubernetes/EKS, comfortable deploying to and reasoning about containerised workloads in a managed K8s environment. Experience orchestrating pipelines with Dagster, Airflow, Prefect, or similar. Solid grasp of ETL concepts and production‐grade pipeline design. Beneficial Skills Interest in chemistry, life sciences, robotics, or manufacturing. Hands ...

TypeScript, React, Next.js Infrastructure & Cloud: AWS (EC2, S3, IAM, Lambda), Docker, Kubernetes; IaC tools Data: Snowflake, Spark, SQL; S3 for raw data ETL & Pipelines: Airflow/Glue Streaming: Kafka or Kinesis Observability: DataDog, Grafana, CloudWatch Security: OAuth2/SAML, encryption, tokenization, zero-trust architecture What You Bring 6+ years ...

TypeScript, React, Next.js Infrastructure & Cloud: AWS (EC2, S3, IAM, Lambda), Docker, Kubernetes; IaC tools Data: Snowflake, Spark, SQL; S3 for raw data ETL & Pipelines: Airflow/Glue Streaming: Kafka or Kinesis Observability: DataDog, Grafana, CloudWatch Security: OAuth2/SAML, encryption, tokenization, zero-trust architecture What You Bring 6+ years ...

data platform. You'll play a key role in building and optimising scalable data pipelines and analytics solutions, working across AWS, Snowflake, dbt and Airflow to deliver high-quality data products. The Company They are a well-established media organisation with a large and diverse audience, using data … models. Your Skills & Experience Strong experience with SQL and modern data engineering practices. Hands-on experience with Snowflake and dbt. Strong experience working with Apache Airflow for data orchestration. Experience working with AWS data services and cloud-based data platforms. Strong Python skills for data engineering and pipeline ...

experience Nice To Have: Experience with Alteryx Server administration, including scheduling, publishing workflows, and managing concurrency Familiarity with workflow orchestration tools such as n8n, Apache Airflow, or Prefect Familiarity with cloud storage environments (AWS S3, SFTP, or similar) Experience with Python or equivalent scripting for lightweight automation tasks ...

isolation Experience with CI/CD, databases, infra-as-code Comfortable debugging production systems Nice to Have Security/compliance (SOC2, HIPAA) Exposure to AirFlow and MLFlow for scientific workflows MSC/PHD in Machine Learning Biotech/pharma experience Why Helical Live with top pharma High ownership, small ...

issuesData Engineers must have:2:1 or above Hons. degree or equivalent in mathematics, computer science or comparable scientific discipline. PythonBuild ETL Pipelines (e.g., Airflow)JavascriptBuild and CI/CD toolsWrite SQL codeCreate and implementing Rest APIsGIT and GIT branchingBI Tools such as Looker, Qlik, Power BI, TableauContainerisation technologiesGoogle ...

notebooks for analysis dbt and Databricks data platform with Delta Lake Lightdash as our BI tool and semantic layer Streamlit for interactive data applications Airflow for orchestration Kubernetes for application deployment CI/CD with GitHub and CircleCI Infrastructure on AWS, deployed with Terraform/Spacelift ...

infrastructure, compute, and storage optimization on AWS or GCP. Strong candidates may also have experience with Data pipelining for AI/ML workflows using Airflow, Prefect, or Dagster. Build system expertise with Bazel, CMake, or distributed build systems. Secrets management tools such as Vault, SOPS, AWS Secrets Manager ...

validation strategies, model evaluation, drift, stability, and monitoring Experience with modern data and ML infrastructure such as AWS, Databricks, MLflow, model registries, model serving, Airflow, or similar orchestration tools Experience building real-time systems, including service design, caching, rate limiting, backpressure, and low-latency architecture Experience building batch pipelines ...

validation strategies, model evaluation, drift, stability, and monitoring Experience with modern data and ML infrastructure such as AWS, Databricks, MLflow, model registries, model serving, Airflow, or similar orchestration tools Experience building real-time systems, including service design, caching, rate limiting, backpressure, and low-latency architecture Experience building batch pipelines ...

validation strategies, model evaluation, drift, stability, and monitoring Experience with modern data and ML infrastructure such as AWS, Databricks, MLflow, model registries, model serving, Airflow, or similar orchestration tools Experience building real-time systems, including service design, caching, rate limiting, backpressure, and low-latency architecture Experience building batch pipelines ...

validation strategies, model evaluation, drift, stability, and monitoring Experience with modern data and ML infrastructure such as AWS, Databricks, MLflow, model registries, model serving, Airflow, or similar orchestration tools Experience building real-time systems, including service design, caching, rate limiting, backpressure, and low-latency architecture Experience building batch pipelines ...

validation strategies, model evaluation, drift, stability, and monitoring Experience with modern data and ML infrastructure such as AWS, Databricks, MLflow, model registries, model serving, Airflow, or similar orchestration tools Experience building real-time systems, including service design, caching, rate limiting, backpressure, and low-latency architecture Experience building batch pipelines ...

validation strategies, model evaluation, drift, stability, and monitoring Experience with modern data and ML infrastructure such as AWS, Databricks, MLflow, model registries, model serving, Airflow, or similar orchestration tools Experience building real-time systems, including service design, caching, rate limiting, backpressure, and low-latency architecture Experience building batch pipelines ...

validation strategies, model evaluation, drift, stability, and monitoring Experience with modern data and ML infrastructure such as AWS, Databricks, MLflow, model registries, model serving, Airflow, or similar orchestration tools Experience building real-time systems, including service design, caching, rate limiting, backpressure, and low-latency architecture Experience building batch pipelines ...

validation strategies, model evaluation, drift, stability, and monitoring Experience with modern data and ML infrastructure such as AWS, Databricks, MLflow, model registries, model serving, Airflow, or similar orchestration tools Experience building real-time systems, including service design, caching, rate limiting, backpressure, and low-latency architecture Experience building batch pipelines ...

validation strategies, model evaluation, drift, stability, and monitoring Experience with modern data and ML infrastructure such as AWS, Databricks, MLflow, model registries, model serving, Airflow, or similar orchestration tools Experience building real-time systems, including service design, caching, rate limiting, backpressure, and low-latency architecture Experience building batch pipelines ...