Lead Software Engineer - LLM Ops Platform Reliability
- Hiring Organisation
- Hackajob Ltd
- Location
- Glasgow, UK
- Employment Type
- Full-time
including model hosting, request routing, continuous batching, and KV-cache optimizationDeploy, host, and lifecycle-manage open-source and proprietary LLMs on Amazon EKS and Amazon SageMaker, as well as on-prem and local GPU clusters, using reproducible infrastructure as code and continuous delivery pipelinesImplement observability (logs, metrics … experience operating LLM inference servers such as vLLM and llm-d (or directly equivalent serving stacks)Hands-on experience hosting and serving LLMs on Amazon EKS and/or Amazon SageMaker, and on local GPU infrastructureKnowledge of LLM reliability and risk considerations, including latency/throughput trade ...