51 to 73 of 73 vLLM Jobs in the UK

built, and the ways they can be exploited. Running agent evaluations in containers or sandboxes, in parallel and at volume. Inference engines such as vLLM or SGLang, for serving the models under evaluation. How we work The role is based in London or San Francisco. We work in the open ...

throughput and cost per token, partnering with the CUDA/GPU engineers. Make the core software architecture calls on serving frameworks and orchestration (e.g. vLLM, TensorRT-LLM, SGLang, Triton Inference Server, or equivalents). Translate throughput, latency, and uptime commitments into concrete technical specifications and serving capacity plans. ...

correctness and maintainability. Python, C++, Rust or other relevant systems or scientific programming experience can provide a route into the role. Familiarity with vLLM or SGLang, simulator frameworks, or hardware performance tools would be useful. Depth in a subset of these areas is sufficient; languages and frameworks will follow ...

Opportunity We are seeking a Senior Software Engineer with a passion for open-source AI infrastructure to work on deploying, extending and optimising vLLM (and potentially other inference serving engines) to support our projects. You will play a crucial, high-impact role across both our key programmes, the Scaling Inference … autonomous agents for software development. What You'll Do Deploy, instrument and monitor open weight models served using vLLM. Implement new features within vLLM to support novel hardware architectures as part of the Scaling Inference Lab. Work with the Panopticon team to identify opportunities to extend vLLM to enhance accuracy ...

models, route tasks to best-fit models, and balance cost, quality, and data sensitivity across workloads. You will set direction for inference engines like vLLM and SGLang and for serving platforms such as KServe. You will work hands-on to wire serving and routing into our platform with observability ...

commonly used databases such as PostgreSQL, Clickhouse, and MongoDB PREFERRED SKILLS AND EXPERIENCE: Experience with LLM inference engines and serving frameworks (e.g., SGLang, TensorRT, vLLM) Experience designing or building with agent SDKs and agent orchestration frameworks Experience with Docker, Kubernetes, and containerized applications Expert knowledge of gRPC (unary, response streaming ...

results against stated application requirements. Practical experience using AI coding agents, with the judgement to validate generated code, data transformations and evaluations. Experience with vLLM, SGLang or similar serving software, long-context workloads, agentic applications, trace analysis or quality evaluation would be valuable. Depth in a subset is sufficient. ...

generation Depth in multi‐agent systems, planning, program synthesis, or retrieval over structured artefacts such as codebases Deep familiarity with the internals of SGLang, vLLM, or comparable inference serving frameworks - scheduler design, memory management, and execution pipelines A background in another field that studies systems of interacting heterogeneous components - neuroscience ...

cost baselines against realistic customer traffic profiles, and tune deployments to hit those targets Deploy and validate new model families on inference frameworks (vLLM, SGLang), determining optimal shapes, quantization configs, and serving patterns across workloads. Model Strategy and Fine‐Tuning Guide customers on model selection, fine‐tuning strategy … issue with an ML engineer in the same afternoon. Preferred Qualifications 10+ years in technical field or engineering roles. Experience with inference serving frameworks (vLLM, SGLang, TensorRT-LLM) and tuning deployments for real workloads. Experience operating as a technical authority inside a customer's environment building within their infrastructure, navigating ...

Location: London, UK Type: Full-time Remote. About Neuphonic Neuphonic is building the next generation of voice AI, making natural, real-time speech accessible across devices, languages, and applications. Our technology enables ultra-low-latency ...

make these capabilities easy for engineering teams to consume while maintaining appropriate controls around security, performance and availability.* Using technologies such as containers, Kubernetes, vLLM and AI gateway platforms, you will deploy and operate scalable inference services, improve observability and performance, and investigate complex technical issues across the platform. … experience in:* Linux, application and server administration* Containers and container orchestration, such as Docker and Kubernetes* AI inference and model serving platforms, such as vLLM, SGLang or similar* AI gateways and API management, such as LiteLLM, Bifrost or similar* GPU workloads, including performance, utilisation and resource management* Programming and scripting ...

make these capabilities easy for engineering teams to consume while maintaining appropriate controls around security, performance and availability. Using technologies such as containers, Kubernetes, vLLM and AI gateway platforms, you will deploy and operate scalable inference services, improve observability and performance, and investigate complex technical issues across the platform. … experience in: Linux, application and server administration Containers and container orchestration, such as Docker and Kubernetes AI inference and model serving platforms, such as vLLM, SGLang or similar AI gateways and API management, such as LiteLLM, Bifrost or similar GPU workloads, including performance, utilisation and resource management Programming and scripting ...

Opportunity We are seeking a Senior Software Engineer with a passion for open-source AI infrastructure to work on deploying, extending and optimising vLLM (and potentially other inference serving engines) to support our projects. You will play a crucial, high-impact role across both our key programmes, the Scaling Inference … autonomous agents for software development. What You'll Do Deploy, instrument and monitor open weight models served using vLLM. Implement new features within vLLM to support novel hardware architectures as part of the Scaling Inference Lab. Work with the Panopticon team to identify opportunities to extend vLLM to enhance accuracy ...

workloads Demonstrated experience building ML applications and models yourself, not just running or deploying existing ones (e.g. serving a pre-built model via vLLM doesn't qualify on its own) Hands-on experience writing GPU kernels in CUDA/C++, or in Triton Low-level experience with model internals e.g. … wrote High ownership and comfort operating in a fast-paced startup environment Nice to have Experience optimizing inference workloads in GPU environments Experience with vLLM or custom inference servers Experience with diffusion models, LLMs, or multimodal architectures more broadly Experience with Kubernetes, Docker, or containerised ML workloads Experience building internal ...

Improve LLM and VLM endpoint latency, throughput, memory efficiency, GPU utilization, quality, and cost per token. Deploy, configure, benchmark, and extend engines such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, and NVIDIA Dynamo. Develop production model-compression workflows covering quantization, quantization-aware training, distillation, low-bit serving, and accuracy … Hands‐on experience deploying or optimizing LLM, VLM, or high‐throughput transformer inference. Practical knowledge of at least one modern inference stack, such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, Ray Serve, KServe, or an equivalent internal system. A strong understanding of transformer inference bottlenecks involving ...

between self-hosted and external models according to the needs of each workload. You will set the technical direction for inference engines such as vLLM and SGLang, and serving platforms such as KServe. You will build foundations that product, delivery and customer teams can rely on. You will own both. … capability should grow and, over time, shape the team around it. Where you need to be strongest You have operated inference engines such as vLLM or SGLang, or serving platforms such as KServe, under production traffic, latency and reliability constraints. You can explain the boundary between serving orchestration and inference ...

checks to simplify adoption and day‐to‐day use Engineer and operate production large language model inference services using modern serving engines such as vLLM, TensorRT‐LLM, SGLang, LLM‐D, or equivalent systems Drive Kubernetes‐based deployment patterns, scaling strategies, networking approaches, and troubleshooting practices to support reliable platform operations … software development kits, libraries, templates, and automated checks to improve platform adoption Deep, hands‐on experience with large language model inference systems such as vLLM, TensorRT‐LLM, SGLang, LLM‐D, or equivalent production serving engines Hands‐on experience building and operating production services on public cloud platforms such ...

during production rollouts. ImproveLLM and VLMendpointlatency, throughput, memory efficiency, GPU utilization, quality, and cost per token. Deploy, configure, benchmark, and extend engines such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server,and NVIDIA Dynamo. Develop production model-compression workflows coveringquantization, quantization-aware training, distillation, low-bit serving, and accuracy recovery. … PyTorch. Hands-on experience deploying or optimizing LLM, VLM, or high-throughput transformerinference. Practical knowledge of at least one modern inferencestack,such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, Ray Serve, KServe, oran equivalent internal system. A strongunderstanding of transformer inferencebottlenecks involvingKV cache, attention, memory bandwidth, batching ...

endpoints for latency, throughput, memory efficiency, GPU utilization, quality, and cost per token. Deploy, configure, benchmark, and extend inference engines such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, or similar systems. Build and productionize model-compression workflows, including quantization, quantization-aware training, distillation, low-bit serving … experience deploying or optimizing LLM, VLM, or high-throughput transformer inference systems. Practical knowledge of at least one modern inference stack such as vLLM, SGLang, TensorRT-LLM, Triton Inference Server, NVIDIA Dynamo, Ray Serve, KServe, or equivalent internal systems. Strong understanding of transformer inference bottlenecks, including KV cache, attention, memory ...

Natural Language Processing (NLP) Researcher We tackle the most complex problems in quantitative finance, bringing scientific clarity to financial complexity from our London HQ. The role involves working with very large text corpora, applying the ...

stack. Investigate efficient execution of workloads across multiple NPUs, including compute/communication overlap and minimizing host-side overhead. Work with AI frameworks like vLLM/SGLang, inference systems and accelerator software stacks to prototype and evaluate new runtime mechanisms. Collaborate with researchers and engineers working across frameworks, compilers, runtime … programming environments such as CUDA, ROCm, OpenCL or CANN/Ascend. Experience with AI frameworks or inference/serving systems such as PyTorch, vLLM, SGLang, ONNX Runtime, or similar systems. Experience with profiling and performance-analysis tools for CPUs, GPUs or NPUs. Knowledge of multi-device or distributed execution, collective ...

production integration. Demonstrate clear latency or throughput gains without unacceptable quality loss. Collaborate with the Inference Foundation team to integrate validated techniques into the vLLM-based serving stack. Partner with pretraining and RL teams when techniques require co-training, retraining, or training auxiliary models to improve inference. Build the right … results, and are eager to take matters into your own hands. Inference or serving experience is a plus, not a requirement. Prior work with vLLM, production LLM deployments, or serving systems will help you ramp up faster. Location and Remote This role is based in our Paris or London offices ...

distributed RL post-training systems, orchestrating trainer, rollout, environment, and reward workloads across thousands of GPUs. Build high-throughput rollout generation, integrating inference engines (vLLM, SGLang), weight synchronization, and asynchronous/off-policy schemes. Design RL environments for agentic, multi-step tasks — sandboxed code execution, tool use, computer use, multimodal … including sandboxed execution and multi-turn tool use. Deep familiarity with RL post-training frameworks (veRL, OpenRLHF, TRL, Ray orchestration) and rollout inference engines (vLLM, SGLang). Strong understanding of GPU clusters, networking, and communication libraries (NCCL, MPI) under mixed training and inference workloads. Nice to Have Running RL training ...