1 to 25 of 33 Permanent vLLM Jobs in England

TensorFlow at a working, hands-on level. Distributed training: Horovod, DeepSpeed, Megatron-LM, or equivalent multi-node training frameworks. Inference & serving: NVIDIA Triton, vLLM, TensorRT-LLM, or equivalent high-throughput serving platforms. MLOps/LLMOps: Kubeflow, MLflow, and at least one hyperscaler ML platform (SageMaker, Azure ML, or Vertex ...

/GenAI platform engineering specifically. You don’t need to tick every box.AI & GenAI Platform Engineering• Model serving and gateway infrastructure (e.g. vLLM, LiteLLM, managed endpoints), with routing, failover, and per-workload cost attribution• Agent orchestration and tool-calling frameworks (e.g. LangGraph or equivalent), including familiarity with the Model Context ...

/ML services (e.g., SageMaker, Vertex AI, AzureML), including sovereign, on‐premise, and hybrid deployment models. Understanding of LLM inference stacks (vLLM, llama.cpp, OpenVINO) and model delivery formats (ONNX, safetensors, HuggingFace model hub). Understanding of benchmarking and LLM performance evaluation (accuracy, latency, throughput). Hands‐on coding skills ...

plus Model fine-tuning practical experience with LoRA/QLoRA, supervised fine-tuning, or RLHF workflows is a plus Inference optimization experience with vLLM, TensorRT-LLM, Triton, or model quantization techniques is desirable Observability tooling practical experience with LLM monitoring, tracing, and evaluation frameworks is a strong plus Familiarity with ...

React, Angular), APIs (REST/GraphQL) Machine Learning: Foundations of MLflow, Weights & Biases, LoRA, PEFT, SageMaker, Vertex AI, AzureML, Kubeflow, Airflow, Prefect, TorchServe, Triton, vLLM, SHAP, LIME, pandas, Spark, SQL LLM tooling: Familiarity with LangChain, Semantic Kernel, prompt chaining, vector search, and context management Performance & observability: Skilled in debugging distributed ...

strongly preferred) Running ML workloads on Kubernetes - GPU scheduling, capacity, and cost management Model serving and inference at production scale (eg KServe, RayServe, Triton, vLLM, or similar) with real latency and cost constraints(preferred RayServe) MLOps pipeline tooling - training pipelines, model registries, feature stores, and lineage (Kubeflow, MLflow, Feast, Weights ...

experience with LLM training cycles, parameter-efficient fine-tuning (PEFT), and sophisticated prompt engineering. Inference Stack: Experience with high-performance inference servers (e.g., vLLM, TGI, or Triton ) and an understanding of how to optimize models for GPU deployment. Infrastructure: Comfortable working in Linux-based environments and proficient in managing containerized ...

install (packaging, versioning, backwards compatibility). Experience with GPU workloads on Kubernetes (device plugins, node scheduling, NVIDIA GPU Operator) and with LLM serving tools (vLLM, Triton, NIM). Familiarity with observability tooling (Prometheus, Grafana, OpenTelemetry) and with exposing it for third‐party components. Exposure to ML orchestration tooling (Flyte, Airflow ...

Architectures: Hands‐on experience building with OpenAI and Anthropic models/APIs; strong knowledge of LLMs and agent architectures. Tooling: Practical experience with LiteLLM, vLLM, Langfuse (or equivalent LLM observability tooling). Experience with Claude Code/Codex and associated coding‐agent workflows is a plus. Evals: Proven ability ...

engineering & synthetic data pipelines Agentic frameworks, reasoning, tool use, memory systems Evaluation & benchmarking (LLM-as-judge, safety/alignment metrics) Inference optimisation (quantization, distillation, vLLM/TensorRT-LLM) Requirements: - PhD or Master's in CS/AI/ML/Physics/Maths or equivalent research experience - Strong hands ...

deployment pipelines and scheduling across clusters and providers. What You Bring Strong Python and system-architecture skills. Experience deploying models with PyTorch, Hugging Face, vLLM, SGLang, TensorRT-LLM, or similar. Experience with queues, scheduling, traffic control, and fleet management at scale. Experience with Linux, Docker, and Kubernetes, and with orchestration ...

also highly valued: Post-graduate degrees and research experience in relevant fields (please list your publications). Deep understanding of inference serving frameworks (e.g. vLLM) Background in statistical analysis Contributions to open source and/or research projects Benefits A collaborative and supportive work environment The opportunity to have ...

with ambiguity and wants ownership. Desirable (or willing to learn) Experience with LLMs: fine-tuning (LoRA/PEFT/full), serving and hosting (e.g. vLLM, TGI, Ollama), and distributed or multi-GPU training. Familiarity with GPUs, cloud, and containers (Docker, etc.). Background in optimisation, Bayesian methods, reinforcement learning ...

with distributed computing frameworks (e.g., Ray, Spark), or specialised batch schedulers/resource managers (e.g., Slurm, Volcano, Kueue). Demonstratedexpertisewith specialised serving engines (e.g.,vLLM,SGLang) or techniques for deploying models in resource-constrained or high-throughput environments. Familiarity withGitOpsand CI/CD (e.g.,ArgoCD,GitHubActions). Experience managing ...

APIs, SDKs, CLIs, notebooks Infrastructure: storage, networking, IAM, observability, infrastructure-as-code Familiarity with tools such as PyTorch, Ray, Slurm, MLflow, Langchain, Hugging Face, vLLM, Triton, or similar. Able to balance performance, cost, usability, flexibility, and reliability. Strong prioritisation, communication, and stakeholder management skills. Comfortable taking ambiguous problems from discovery ...

pragmatism to prioritise what matters in a fast-moving environment. Bonus: experience with GPU clusters or ML training/inference infrastructure (e.g. vLLM, Slurm), on-prem/bare-metal or deep networking experience, or open-source contributions in the cloud-native community. Location This role is remote ...

optimizing models in PyTorch or JAX Deep knowledge of at least one of these areas: LLMs: pretraining, alignment, evals GPU inference and performance: vLLM, Triton, CUDA, profiling Reinforcement learning (RL) Scientific ML: weather and climate, physics, biology Able to explain why a problem is hard for a frontier model Nice ...

Computer Science, Electronic Engineering, Maths, Physics, or related degree and 5+ years of industry experience. You may also have: Experience with inference serving stacks (vLLM, SGLang, TensorRT-LLM, Triton) or ML compiler and runtime toolchains. Experience working with GPUs, TPUs, or other machine learning accelerators. A track record of contributing ...

management in development tools and software including CLIs, SDKs and APIs. Knowledge of machine learning frameworks, runtimes, infrastructure such as PyTorch, ONNX, ExecuTorch, Llama.cpp, vLLM and LiteRT. Understanding of model deployment to edge, embedded or heterogeneous computing, with an understanding of trade-offs between accuracy, performance and power consumption. ...

transformer inference, including prefill versus decode, KV‐cache behaviour, attention variants and performance bottlenecks Hands‐on experience with LLM serving frameworks such as vLLM, SGLang, TensorRT‐LLM or TGI, and the PyTorch ecosystem Experience with inference optimisation techniques, including quantisation, speculative decoding and model parallelism across modern GPU architectures Strong ...

post-training Experience with distributed training and GPU systems Strong experimental judgement Ability to work comfortably across ML research and systems engineering Optional vLLM, SGLang, Ray, FSDP or Megatron-LM CUDA, Triton, CuTE or GPU performance optimisation Shortlisted candidates will be contacted within 48 hours. #J-18808-Ljbffr ...

Skills and Experience : Experience using software simulators and/or FPGAs Exposure to any of the following: driver development, inference engines like llama.cpp or vLLM, performance analysis of ML and GenAI workloads, knowledge of Neural Network Processing Units (NPU) or Graphics Processing Units (GPU) and how they are used ...

throughput and cost per token, partnering with the CUDA/GPU engineers Make the core software architecture calls on serving frameworks and orchestration (e.g. vLLM, TensorRT-LLM, SGLang, Triton Inference Server or equivalents) Translate throughput, latency and uptime commitments into concrete technical specifications and serving capacity plans Act as direct ...

throughput and cost per token, partnering with the CUDA/GPU engineers. Make the core software architecture calls on serving frameworks and orchestration (e.g. vLLM, TensorRT-LLM, SGLang, Triton Inference Server, or equivalents). Translate throughput, latency, and uptime commitments into concrete technical specifications and serving capacity plans. ...

Opportunity We are seeking a Senior Software Engineer with a passion for open-source AI infrastructure to work on deploying, extending and optimising vLLM (and potentially other inference serving engines) to support our projects. You will play a crucial, high-impact role across both our key programmes, the Scaling Inference … autonomous agents for software development. What You'll Do Deploy, instrument and monitor open weight models served using vLLM. Implement new features within vLLM to support novel hardware architectures as part of the Scaling Inference Lab. Work with the Panopticon team to identify opportunities to extend vLLM to enhance accuracy ...