1 of 1 Amazon EKS Jobs in Glasgow

model hosting, request routing, continuous batching, and KV‐cache optimization Deploy, host, and lifecycle‐manage open‐source and proprietary LLMs on Amazon EKS and Amazon SageMaker, as well as on‐prem and local GPU clusters, using reproducible infrastructure as code and continuous delivery pipelines Implement … inference servers such as vLLM and llm‐d (or directly equivalent serving stacks) Hands‐on experience hosting and serving LLMs on Amazon EKS and/or Amazon SageMaker, and on local GPU infrastructure Knowledge of LLM reliability and risk considerations, including latency/throughput trade ...