26 to 50 of 58 vLLM Jobs in England

design, operational risk and engineering trade-offs. Fluent Python and daily use of coding agents. NICE TO HAVE Inference engines such as SGLang or vLLM, and RL training frameworks such as veRL, Miles or TRL. Agent environment and evaluation frameworks such as Harbor. Sandboxed execution for agents, such as Firecracker ...

training frameworks such as veRL, Miles or TRL, RL environment frameworks such as Harbor or Prime Intellect's verifiers, and inference engines such as vLLM or SGLang. Building agent harnesses and long-horizon tasks with tool use, such as search, code execution and external APIs. Designing learning signals for long ...

with hardware/software co-design, or with hardware performance modelling and simulation. Contributions to performance-critical open-source projects (e.g. PyTorch, Triton, JAX, vLLM). A collaborative and supportive work environment The opportunity to have a high impact in a growing organisation Competitive salary package and pension Professional development ...

with distributed computing frameworks (e.g., Ray, Spark), or specialised batch schedulers/resource managers (e.g., Slurm, Volcano, Kueue). Demonstratedexpertisewith specialised serving engines (e.g.,vLLM,SGLang) or techniques for deploying models in resource-constrained or high-throughput environments. Familiarity withGitOpsand CI/CD (e.g.,ArgoCD,GitHubActions). Experience managing ...

management in development tools and software including CLIs, SDKs and APIs. Knowledge of machine learning frameworks, runtimes, infrastructure such as PyTorch, ONNX, ExecuTorch, Llama.cpp, vLLM and LiteRT. Understanding of model deployment to edge, embedded or heterogeneous computing, with an understanding of trade-offs between accuracy, performance and power consumption. ...

also be interested to hear about any experience you have in the following areas: Experience with ML frameworks and LLM inference , such as PyTorch, vLLM, SGLang, llama.cpp or similar Experience analysing and optimizing CPU performance in heterogeneous CPU + accelerator systems Familiarity with agentic AI frameworks and protocols Understanding ...

engineering experience, ideally with hands‐on work in inference optimization, model training, GPU programming, or ML infrastructure You’re familiar with the serving (e.g., vLLM, SGLang) and training (e.g., slime, verl, TRL) toolchains and can go deep on at least one (you don’t need all of them ...

Skills and Experience : Experience using software simulators and/or FPGAs Exposure to any of the following: driver development, inference engines like llama.cpp or vLLM, performance analysis of ML and GenAI workloads, knowledge of Neural Network Processing Units (NPU) or Graphics Processing Units (GPU) and how they are used ...

throughput-optimized setups. Driving consistent “playbook” style deployments for common models and use cases. Become a power user of different runtimes such as vLLM, SGLang, and TRT-LLM and all the common configurations and tradeoffs between them Drive POC and project execution , including: Scoping POCs and keeping stakeholders aligned ...

other image-intelligence work.* Text recognition in low-quality images and video, including non-Latin scripts.* Serving open-weight models efficiently in production, with vLLM or similar.* Distributed training across multiple GPUs.## What you'll getWe're a small, performance-driven team at one of the fast-growing startups ...

post-training Experience with distributed training and GPU systems Strong experimental judgement Ability to work comfortably across ML research and systems engineering Optional vLLM, SGLang, Ray, FSDP or Megatron-LM CUDA, Triton, CuTE or GPU performance optimisation Shortlisted candidates will be contacted within 48 hours. #J-18808-Ljbffr ...

commonly used databases such as PostgreSQL, Clickhouse, and MongoDB PREFERRED SKILLS AND EXPERIENCE: Experience with LLM inference engines and serving frameworks (e.g., SGLang, TensorRT, vLLM) Experience designing or building with agent SDKs and agent orchestration frameworks Experience with Docker, Kubernetes, and containerized applications Expert knowledge of gRPC (unary, response streaming ...

transformer inference, including prefill versus decode, KV‐cache behaviour, attention variants and performance bottlenecks Hands‐on experience with LLM serving frameworks such as vLLM, SGLang, TensorRT‐LLM or TGI, and the PyTorch ecosystem Experience with inference optimisation techniques, including quantisation, speculative decoding and model parallelism across modern GPU architectures Strong ...

such as PostgreSQL, Clickhouse, and MongoDB Knowledge of service observability and reliability best practicesExperience with LLM inference engines and serving frameworks (e.g., SGLang, TensorRT, vLLM) Experience designing or building with agent SDKs and agent orchestration frameworks Experience with Docker, Kubernetes, and containerized applications Expert knowledge of gRPC (unary, response streaming ...

throughput and cost per token, partnering with the CUDA/GPU engineers Make the core software architecture calls on serving frameworks and orchestration (e.g. vLLM, TensorRT-LLM, SGLang, Triton Inference Server or equivalents) Translate throughput, latency and uptime commitments into concrete technical specifications and serving capacity plans Act as direct ...

built, and the ways they can be exploited. Running agent evaluations in containers or sandboxes, in parallel and at volume. Inference engines such as vLLM or SGLang, for serving the models under evaluation. How we work The role is based in London or San Francisco. We work in the open ...

throughput and cost per token, partnering with the CUDA/GPU engineers. Make the core software architecture calls on serving frameworks and orchestration (e.g. vLLM, TensorRT-LLM, SGLang, Triton Inference Server, or equivalents). Translate throughput, latency, and uptime commitments into concrete technical specifications and serving capacity plans. ...

correctness and maintainability. Python, C++, Rust or other relevant systems or scientific programming experience can provide a route into the role. Familiarity with vLLM or SGLang, simulator frameworks, or hardware performance tools would be useful. Depth in a subset of these areas is sufficient; languages and frameworks will follow ...

models, route tasks to best-fit models, and balance cost, quality, and data sensitivity across workloads. You will set direction for inference engines like vLLM and SGLang and for serving platforms such as KServe. You will work hands-on to wire serving and routing into our platform with observability ...

commonly used databases such as PostgreSQL, Clickhouse, and MongoDB PREFERRED SKILLS AND EXPERIENCE: Experience with LLM inference engines and serving frameworks (e.g., SGLang, TensorRT, vLLM) Experience designing or building with agent SDKs and agent orchestration frameworks Experience with Docker, Kubernetes, and containerized applications Expert knowledge of gRPC (unary, response streaming ...

results against stated application requirements. Practical experience using AI coding agents, with the judgement to validate generated code, data transformations and evaluations. Experience with vLLM, SGLang or similar serving software, long-context workloads, agentic applications, trace analysis or quality evaluation would be valuable. Depth in a subset is sufficient. ...

generation Depth in multi‐agent systems, planning, program synthesis, or retrieval over structured artefacts such as codebases Deep familiarity with the internals of SGLang, vLLM, or comparable inference serving frameworks - scheduler design, memory management, and execution pipelines A background in another field that studies systems of interacting heterogeneous components - neuroscience ...

cost baselines against realistic customer traffic profiles, and tune deployments to hit those targets Deploy and validate new model families on inference frameworks (vLLM, SGLang), determining optimal shapes, quantization configs, and serving patterns across workloads. Model Strategy and Fine‐Tuning Guide customers on model selection, fine‐tuning strategy … issue with an ML engineer in the same afternoon. Preferred Qualifications 10+ years in technical field or engineering roles. Experience with inference serving frameworks (vLLM, SGLang, TensorRT-LLM) and tuning deployments for real workloads. Experience operating as a technical authority inside a customer's environment building within their infrastructure, navigating ...

Location: London, UK Type: Full-time Remote. About Neuphonic Neuphonic is building the next generation of voice AI, making natural, real-time speech accessible across devices, languages, and applications. Our technology enables ultra-low-latency ...

Opportunity We are seeking a Senior Software Engineer with a passion for open-source AI infrastructure to work on deploying, extending and optimising vLLM (and potentially other inference serving engines) to support our projects. You will play a crucial, high-impact role across both our key programmes, the Scaling Inference … autonomous agents for software development. What You'll Do Deploy, instrument and monitor open weight models served using vLLM. Implement new features within vLLM to support novel hardware architectures as part of the Scaling Inference Lab. Work with the Panopticon team to identify opportunities to extend vLLM to enhance accuracy ...