1 of 1 Open Source Jobs in Renfrewshire

serving infrastructure (such as vLLM and llm-d), including model hosting, request routing, continuous batching, and KV-cache optimization Deploy, host, and lifecycle-manage open-source and proprietary LLMs on Amazon EKS and Amazon SageMaker, as well as on-prem and local GPU clusters, using reproducible infrastructure … Amazon Bedrock for managed model hosting Experience with online LLM quality monitoring (e.g., hallucination, toxicity, drift detection) and tracing via OpenTelemetry conventions Contributions to open-source LLM serving or inference projects (e.g., vLLM, llm-d, Ray, KServe, Triton) ABOUT US J.P. Morgan is a global leader in financial ...