2 of 2 vLLM Jobs in Edinburgh

field. Strong knowledge of distributed systems, operating systems,machine learning systems architecture, Inference serving, and AI Infrastructure. Hands-on experience withLLM serving frameworks(e.g.,vLLM,Ray Serve,TensorRT-LLM,TGI) anddistributed KV cache optimization. Proficiency inC/C++, with additional experience inPythonfor research prototyping. Solid grounding insystems research methodology,distributed ...

make these capabilities easy for engineering teams to consume while maintaining appropriate controls around security, performance and availability.* Using technologies such as containers, Kubernetes, vLLM and AI gateway platforms, you will deploy and operate scalable inference services, improve observability and performance, and investigate complex technical issues across the platform. … experience in:* Linux, application and server administration* Containers and container orchestration, such as Docker and Kubernetes* AI inference and model serving platforms, such as vLLM, SGLang or similar* AI gateways and API management, such as LiteLLM, Bifrost or similar* GPU workloads, including performance, utilisation and resource management* Programming and scripting ...