626 to 650 of 887 Apache Airflow Jobs

engines. Strong Java or Python - ideally both, as our stream processing is Java and our orchestration is Python. Hands-on production stream processing experience (Apache Flink, Kafka Streams, Spark Structured Streaming, or Beam), with the ability to reason about keyed state, watermarks, checkpointing, and event-time versus processing-time … semantics. Practical Apache Kafka knowledge beyond consumer usage - partitioning, compaction, consumer group behavior, and schema evolution (Avro/Schema Registry). Experience with a workflow or ETL orchestration tool - Dagster preferred, or a comparable orchestrator such as Airflow or Prefect. SQL fluency and production experience with a columnar ...

engines. Strong Java or Python - ideally both, as our stream processing is Java and our orchestration is Python. Hands-on production stream processing experience (Apache Flink, Kafka Streams, Spark Structured Streaming, or Beam), with the ability to reason about keyed state, watermarks, checkpointing, and event-time versus processing-time … semantics. Practical Apache Kafka knowledge beyond consumer usage - partitioning, compaction, consumer group behavior, and schema evolution (Avro/Schema Registry). Experience with a workflow or ETL orchestration tool - Dagster preferred, or a comparable orchestrator such as Airflow or Prefect. SQL fluency and production experience with a columnar ...

/CD experience. Highly Desirable Familiarity with OneTick, PostgreSQL, ClickHouse, Snowflake, Databricks, kdb+ or similar. Order book reconstruction Tick-to-bar aggregation Airflow, Dagster, Prefect or MLflow Experience supporting quantitative research or trading systems Multi-asset market data (Equities, Futures, FX, Options, ETFs, CFDs) What We're Looking … market data engineering with proven experience building reliable, scalable data platforms. Our non-negotiable requirements are: Expert Python Strong SQL Scalable data pipeline development Apache Parquet Market data normalisation Automated data quality controls Deep tick and market data expertise Experience in MLOps is advantageous but not essential. We welcome ...

related role. Bonus Points: Experience with Google Cloud Platform (GCP) data services (BigQuery, Dataflow, Pub/Sub, Cloud Storage). Experience with Apache Beam. Background in building infrastructure for AI/ML workloads or cloud computing platforms. Experience on a growing or early-stage data team, contributing to foundational … development. Familiarity with data orchestration tools (e.g., Airflow, Dagster, Prefect). Experience with real-time data processing and streaming architectures. Benefits: Competitive compensation Restricted Stock Units Paid time off & paid holidays Comprehensive health, dental & vision insurance Employer contributions to HSA account Paid parental leave Paid life insurance, short-term ...

data engineering Experience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal) Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governance Practical cloud-native experience (AWS, Azure, or GCP) Ability … hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with emerging technologies in data engineering ...

data engineering Experience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal) Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governance Practical cloud-native experience (AWS, Azure, or GCP) Ability … hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with emerging technologies in data engineering ...

/ML, data engineering) Experience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal) Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governance Practical cloud-native experience (AWS, Azure … hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with AI evaluation and observability practices ...

/ML, data engineering) Experience in large-scale data processing, microservices, API design, Kafka, Redis, MemCached, observability tools (Dynatrace, Splunk, Grafana), and orchestration frameworks (Airflow, Temporal) Advanced working knowledge of relational and NoSQL databases, vector stores, data lake architectures, and data governance Practical cloud-native experience (AWS, Azure … hands-on experience with Spark/PySpark and other big data processing technologies Expertise in open-source table formats and catalog services such as Apache Iceberg Experience with LLM orchestration frameworks and model serving infrastructure or managed endpoints (AWS Bedrock, Azure OpenAI) Familiarity with AI evaluation and observability practices ...

including model deployment, monitoring, evaluation, and lifecycle management Strong hands‐on understanding of modern data and AI tooling, ideally including SQL, Python, Snowflake, dbt, Airflow, Metabase, Tableau, and AWS‐based data services Experience with data ingestion, streaming, and change data capture tooling, such as Fivetran, Estuary, or similar platforms … standards, and enablement for analytics engineers Ability to oversee API‐based and batch data extraction processes, particularly those built in Python and orchestrated using Airflow Understanding of machine learning tooling and lifecycle management, ideally including DVC, MLflow, Evidently AI, or equivalent platforms for experiment tracking, versioning, monitoring, and evaluation ...

use. Coordinate connectivity and firewall requests with upstream data teams and platform teams. Support infrastructure needs for new data onboarding and scheduler maintenance with Airflow and event triggers. Perform other duties as required. Experience Requirements: Expert experience with AWS services including EC2, S3, IAM, VPC, Glue, Athena, EMR, Secrets … integration. Integration against REST APIs and cloud SDKs such as boto3. Preferred: graph databases such as AWS Neptune or Neo4j. Preferred: Apache Kafka or similar streaming platforms. Preferred: Apache Spark with Scala for distributed and streaming processing. Preferred: Airflow or similar orchestration platforms. Preferred: telecommunications or large ...

use. Coordinate connectivity and firewall requests with upstream data teams and platform teams. Support infrastructure needs for new data onboarding and scheduler maintenance with Airflow and event triggers. Perform other duties as required. Experience Requirements: Expert experience with AWS services including EC2, S3, IAM, VPC, Glue, Athena, EMR, Secrets … integration. Integration against REST APIs and cloud SDKs such as boto3. Preferred: graph databases such as AWS Neptune or Neo4j. Preferred: Apache Kafka or similar streaming platforms. Preferred: Apache Spark with Scala for distributed and streaming processing. Preferred: Airflow or similar orchestration platforms. Preferred: telecommunications or large ...

. Experience with customer-facing, productized APIs and the platform underpinnings that support them is a plus. Familiarity with modern workflow orchestration tools (e.g. Apache Airflow, Temporal, etc) for managing and automating data or processing pipelines is a plus. Self-starter with the ability to work as part ...

. Experience with customer-facing, productized APIs and the platform underpinnings that support them is a plus. Familiarity with modern workflow orchestration tools (e.g. Apache Airflow, Temporal, etc) for managing and automating data or processing pipelines is a plus. Self-starter with the ability to work as part ...

. Experience with customer-facing, productized APIs and the platform underpinnings that support them is a plus. Familiarity with modern workflow orchestration tools (e.g. Apache Airflow, Temporal, etc) for managing and automating data or processing pipelines is a plus. Self-starter with the ability to work as part ...

. Experience with customer-facing, productized APIs and the platform underpinnings that support them is a plus. Familiarity with modern workflow orchestration tools (e.g. Apache Airflow, Temporal, etc) for managing and automating data or processing pipelines is a plus. Self-starter with the ability to work as part ...

. Experience with customer-facing, productized APIs and the platform underpinnings that support them is a plus. Familiarity with modern workflow orchestration tools (e.g. Apache Airflow, Temporal, etc) for managing and automating data or processing pipelines is a plus. Self-starter with the ability to work as part ...

. Experience with customer-facing, productized APIs and the platform underpinnings that support them is a plus. Familiarity with modern workflow orchestration tools (e.g. Apache Airflow, Temporal, etc) for managing and automating data or processing pipelines is a plus. Self-starter with the ability to work as part ...

. Experience with customer-facing, productized APIs and the platform underpinnings that support them is a plus. Familiarity with modern workflow orchestration tools (e.g. Apache Airflow, Temporal, etc) for managing and automating data or processing pipelines is a plus. Self-starter with the ability to work as part ...

. Experience with customer-facing, productized APIs and the platform underpinnings that support them is a plus. Familiarity with modern workflow orchestration tools (e.g. Apache Airflow, Temporal, etc) for managing and automating data or processing pipelines is a plus. Self-starter with the ability to work as part ...

. Experience with customer-facing, productized APIs and the platform underpinnings that support them is a plus. Familiarity with modern workflow orchestration tools (e.g. Apache Airflow, Temporal, etc) for managing and automating data or processing pipelines is a plus. Self-starter with the ability to work as part ...

. Experience with customer-facing, productized APIs and the platform underpinnings that support them is a plus. Familiarity with modern workflow orchestration tools (e.g. Apache Airflow, Temporal, etc) for managing and automating data or processing pipelines is a plus. Self-starter with the ability to work as part ...

. Experience with customer-facing, productized APIs and the platform underpinnings that support them is a plus. Familiarity with modern workflow orchestration tools (e.g. Apache Airflow, Temporal, etc) for managing and automating data or processing pipelines is a plus. Self-starter with the ability to work as part ...

. Experience with customer-facing, productized APIs and the platform underpinnings that support them is a plus. Familiarity with modern workflow orchestration tools (e.g. Apache Airflow, Temporal, etc) for managing and automating data or processing pipelines is a plus. Self-starter with the ability to work as part ...

. Experience with customer-facing, productized APIs and the platform underpinnings that support them is a plus. Familiarity with modern workflow orchestration tools (e.g. Apache Airflow, Temporal, etc) for managing and automating data or processing pipelines is a plus. Self-starter with the ability to work as part ...

. Experience with customer-facing, productized APIs and the platform underpinnings that support them is a plus. Familiarity with modern workflow orchestration tools (e.g. Apache Airflow, Temporal, etc) for managing and automating data or processing pipelines is a plus. Self-starter with the ability to work as part ...