26 to 43 of 43 vLLM Jobs in London

commonly used databases such as PostgreSQL, Clickhouse, and MongoDB PREFERRED SKILLS AND EXPERIENCE: Experience with LLM inference engines and serving frameworks (e.g., SGLang, TensorRT, vLLM) Experience designing or building with agent SDKs and agent orchestration frameworks Experience with Docker, Kubernetes, and containerized applications Expert knowledge of gRPC (unary, response streaming ...

transformer inference, including prefill versus decode, KV‐cache behaviour, attention variants and performance bottlenecks Hands‐on experience with LLM serving frameworks such as vLLM, SGLang, TensorRT‐LLM or TGI, and the PyTorch ecosystem Experience with inference optimisation techniques, including quantisation, speculative decoding and model parallelism across modern GPU architectures Strong ...

such as PostgreSQL, Clickhouse, and MongoDB Knowledge of service observability and reliability best practicesExperience with LLM inference engines and serving frameworks (e.g., SGLang, TensorRT, vLLM) Experience designing or building with agent SDKs and agent orchestration frameworks Experience with Docker, Kubernetes, and containerized applications Expert knowledge of gRPC (unary, response streaming ...

throughput and cost per token, partnering with the CUDA/GPU engineers Make the core software architecture calls on serving frameworks and orchestration (e.g. vLLM, TensorRT-LLM, SGLang, Triton Inference Server or equivalents) Translate throughput, latency and uptime commitments into concrete technical specifications and serving capacity plans Act as direct ...

built, and the ways they can be exploited. Running agent evaluations in containers or sandboxes, in parallel and at volume. Inference engines such as vLLM or SGLang, for serving the models under evaluation. How we work The role is based in London or San Francisco. We work in the open ...

throughput and cost per token, partnering with the CUDA/GPU engineers. Make the core software architecture calls on serving frameworks and orchestration (e.g. vLLM, TensorRT-LLM, SGLang, Triton Inference Server, or equivalents). Translate throughput, latency, and uptime commitments into concrete technical specifications and serving capacity plans. ...

models, route tasks to best-fit models, and balance cost, quality, and data sensitivity across workloads. You will set direction for inference engines like vLLM and SGLang and for serving platforms such as KServe. You will work hands-on to wire serving and routing into our platform with observability ...

commonly used databases such as PostgreSQL, Clickhouse, and MongoDB PREFERRED SKILLS AND EXPERIENCE: Experience with LLM inference engines and serving frameworks (e.g., SGLang, TensorRT, vLLM) Experience designing or building with agent SDKs and agent orchestration frameworks Experience with Docker, Kubernetes, and containerized applications Expert knowledge of gRPC (unary, response streaming ...

generation Depth in multi‐agent systems, planning, program synthesis, or retrieval over structured artefacts such as codebases Deep familiarity with the internals of SGLang, vLLM, or comparable inference serving frameworks - scheduler design, memory management, and execution pipelines A background in another field that studies systems of interacting heterogeneous components - neuroscience ...

cost baselines against realistic customer traffic profiles, and tune deployments to hit those targets Deploy and validate new model families on inference frameworks (vLLM, SGLang), determining optimal shapes, quantization configs, and serving patterns across workloads. Model Strategy and Fine‐Tuning Guide customers on model selection, fine‐tuning strategy … issue with an ML engineer in the same afternoon. Preferred Qualifications 10+ years in technical field or engineering roles. Experience with inference serving frameworks (vLLM, SGLang, TensorRT-LLM) and tuning deployments for real workloads. Experience operating as a technical authority inside a customer's environment building within their infrastructure, navigating ...

Location: London, UK Type: Full-time Remote. About Neuphonic Neuphonic is building the next generation of voice AI, making natural, real-time speech accessible across devices, languages, and applications. Our technology enables ultra-low-latency ...

Improve LLM and VLM endpoint latency, throughput, memory efficiency, GPU utilization, quality, and cost per token. Deploy, configure, benchmark, and extend engines such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, and NVIDIA Dynamo. Develop production model-compression workflows covering quantization, quantization-aware training, distillation, low-bit serving, and accuracy … Hands‐on experience deploying or optimizing LLM, VLM, or high‐throughput transformer inference. Practical knowledge of at least one modern inference stack, such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, Ray Serve, KServe, or an equivalent internal system. A strong understanding of transformer inference bottlenecks involving ...

between self-hosted and external models according to the needs of each workload. You will set the technical direction for inference engines such as vLLM and SGLang, and serving platforms such as KServe. You will build foundations that product, delivery and customer teams can rely on. You will own both. … capability should grow and, over time, shape the team around it. Where you need to be strongest You have operated inference engines such as vLLM or SGLang, or serving platforms such as KServe, under production traffic, latency and reliability constraints. You can explain the boundary between serving orchestration and inference ...

during production rollouts. ImproveLLM and VLMendpointlatency, throughput, memory efficiency, GPU utilization, quality, and cost per token. Deploy, configure, benchmark, and extend engines such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server,and NVIDIA Dynamo. Develop production model-compression workflows coveringquantization, quantization-aware training, distillation, low-bit serving, and accuracy recovery. … PyTorch. Hands-on experience deploying or optimizing LLM, VLM, or high-throughput transformerinference. Practical knowledge of at least one modern inferencestack,such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, Ray Serve, KServe, oran equivalent internal system. A strongunderstanding of transformer inferencebottlenecks involvingKV cache, attention, memory bandwidth, batching ...

endpoints for latency, throughput, memory efficiency, GPU utilization, quality, and cost per token. Deploy, configure, benchmark, and extend inference engines such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, or similar systems. Build and productionize model-compression workflows, including quantization, quantization-aware training, distillation, low-bit serving … experience deploying or optimizing LLM, VLM, or high-throughput transformer inference systems. Practical knowledge of at least one modern inference stack such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, Ray Serve, KServe, or equivalent internal systems. Strong understanding of transformer inference bottlenecks, including KV cache, attention, memory ...

Natural Language Processing (NLP) Researcher We tackle the most complex problems in quantitative finance, bringing scientific clarity to financial complexity from our London HQ. The role involves working with very large text corpora, applying the ...

production integration. Demonstrate clear latency or throughput gains without unacceptable quality loss. Collaborate with the Inference Foundation team to integrate validated techniques into the vLLM-based serving stack. Partner with pretraining and RL teams when techniques require co-training, retraining, or training auxiliary models to improve inference. Build the right … results, and are eager to take matters into your own hands. Inference or serving experience is a plus, not a requirement. Prior work with vLLM, production LLM deployments, or serving systems will help you ramp up faster. Location and Remote This role is based in our Paris or London offices ...

distributed RL post-training systems, orchestrating trainer, rollout, environment, and reward workloads across thousands of GPUs. Build high-throughput rollout generation, integrating inference engines (vLLM, SGLang), weight synchronization, and asynchronous/off-policy schemes. Design RL environments for agentic, multi-step tasks — sandboxed code execution, tool use, computer use, multimodal … including sandboxed execution and multi-turn tool use. Deep familiarity with RL post-training frameworks (veRL, OpenRLHF, TRL, Ray orchestration) and rollout inference engines (vLLM, SGLang). Strong understanding of GPU clusters, networking, and communication libraries (NCCL, MPI) under mixed training and inference workloads. Nice to Have Running RL training ...