2 of 2 Dagster Jobs in Leicestershire

build and why. You'll own the full lifecycle of key Python integrations, driving architecture, performance, and feature direction across: Orchestration Platforms: Apache Airflow, Dagster, Prefect Transformation Tools: dbt, SQLMesh AI & LLM Ecosystem: LangChain, LlamaIndex, n8n, and broader AI tooling: embedding pipelines, retrieval-augmented generation with ClickHouse … ClickHouse's Python connector and SDK ecosystem, raising the bar on performance, reliability, and API design Build and maintain integrations with orchestration platforms (Airflow, Dagster, Prefect) and transformation tools (dbt) to enterprise-grade quality standards Drive the AI/LLM integration strategy: designing connectors and patterns that make ClickHouse ...

ingesting, transforming, and storing large volumes of data. Implement scalable, fault-tolerant systems using modern frameworks such as Spark, Temporal, Airflow o r Dagster . Develop APIs and services that expose reliable data access, transformation, and orchestration capabilities. Contribute to geospatial data processing workflows using tools such as Apache … Spark SQL preferred); experience with distributed data processing frameworks such as Spark, Flink, or Beam. Familiarity with orchestration frameworks (e.g., Airflow, Temporal, Dagster ) and workflow automation principles. Experience with cloud platforms (AWS, GCP, or Azure) and containerization tools (Docker, Kubernetes). Exposure to batch and streaming architectures and understanding ...