4 of 4 Permanent Ray Jobs in the UK

knowledge of distributed systems, operating systems, machine learning systems architecture, Inference serving, and AI Infrastructure. · Hands-on experience with LLM serving frameworks (e.g., vLLM, Ray Serve, TensorRT-LLM, TGI) and distributed KV cache optimization. · Proficiency in C/C++, with additional experience in Python for research prototyping. · Solid grounding ...

optimized Python. You know how to profile code and squeeze every ounce of performance out of NVIDIA GPUs. Distributed Systems & Scaling. Experience with Kubernetes, Ray, custom load balancing, multi-GPU/multi-node inference, and reliably handling thousands of concurrent connections. Public work. Non-trivial systems programming projects, open-source ...

about twenty machine learning scientists. The team is supported by an ML Ops team that provides state-of-the-art tooling (including AWS, Encord, Ray, PyTorch Lightning and Weights & Biases). The Applied Science team works closely with product engineering to deploy models to serve our worldwide customer base. Position ...

large-scale inference and data pipelines, focusing on KV cache management, heterogeneous memory scheduling, and high-throughput inference serving using frameworks like vLLM, Ray Serve, and modern PyTorch Distributed systems. Scalable Model Serving Infrastructure : Develop and evaluate frameworks that enable efficient multi-tenant, low-latency, and fault-tolerant AI serving … computing, or large-scale AI infrastructure are also welcome At least 2 years of experience with LLM inference/serving framework optimization (vLLM/Ray Serve/TensorRT-LLM/PyTorch) Hands-on experience with distributed KV cache optimization Familiarity with GPU and how they execute LLMs Strong knowledge ...