18 of 18 Permanent vLLM Jobs in London

TensorFlow at a working, hands-on level. Distributed training: Horovod, DeepSpeed, Megatron-LM, or equivalent multi-node training frameworks. Inference & serving: NVIDIA Triton, vLLM, TensorRT-LLM, or equivalent high-throughput serving platforms. MLOps/LLMOps: Kubeflow, MLflow, and at least one hyperscaler ML platform (SageMaker, Azure ML, or Vertex ...

plus Model fine-tuning practical experience with LoRA/QLoRA, supervised fine-tuning, or RLHF workflows is a plus Inference optimization experience with vLLM, TensorRT-LLM, Triton, or model quantization techniques is desirable Observability tooling practical experience with LLM monitoring, tracing, and evaluation frameworks is a strong plus Familiarity with ...

development practices (testing, code review, CI/CD). Knowledge of vector databases (Pinecone, Weaviate, Qdrant) and embedding models. Experience with model serving frameworks (vLLM, TensorRT, Ray). Experience with A/B testing and experimentation frameworks for AI features. Experience with model observability tools (LangSmith, W&B, MLflow). ...

Hands-on experience with cloud-native ML infrastructure platforms Knowledge of vector databases (Pinecone, Weaviate, Qdrant) and embedding models Experience with model serving frameworks (vLLM, TensorRT, Ray) Experience with A/B testing and experimentation frameworks for AI features Contributions to open-source ML projects or research publications Experience with ...

preference. Technical Expertise: Deep proficiency in Python and modern software development practices. Hands‐on experience with Distributed Training infrastructure (Multi‐node GPU training, Kubernetes, vLLM). Familiarity with Deep Learning frameworks (PyTorch). Experience with MLOps tools and experiment tracking (e.g., ClearML, MLFlow, Weights & Biases). Research Fluency: Ability ...

software development lifecycle, including designing, generating code, testing, monitoring and releasing software Nice To Haves Experience with LLM inference engines and serving frameworks (e.g., vLLM, SGLang, TensorRT-LLM) in production Experience with distributed/multi‐node fine‐tuning and training pipelines (SFT, DPO/RLHF, LoRA), including data preparation ...

experience with LLM training cycles, parameter-efficient fine-tuning (PEFT), and sophisticated prompt engineering. Inference Stack: Experience with high-performance inference servers (e.g., vLLM, TGI, or Triton ) and an understanding of how to optimize models for GPU deployment. Infrastructure: Comfortable working in Linux-based environments and proficient in managing containerized ...

without degrading their reasoning capabilities. Experience with machine translation, multilingual NLP, or language quality estimation. Familiarity with inference and serving at scale (e.g. via vLLM, SGLang, TensorRT‐LLM, etc) and long‐context modelling. Publications at top‐tier venues. What we offer Diverse and internationally distributed team : joining our team means ...

like across different GPU generations and topologies. Exposure to MLOps tooling and AI platform layers: experiment tracking (MLflow, W'B), model serving frameworks (Triton, vLLM), and pipeline orchestration (Kubeflow, Airflow). Familiarity with InfiniBand and high-performance networking as it relates to distributed training performance — sufficient to engage credibly with ...

across the stack, for example py‐spy, PyTorch profiler, Nsight, perf, tracing, metrics, logs, or custom instrumentation Have experience with inference stacks such as vLLM, SGLang, TensorRT‐LLM, Dynamo, or custom serving infrastructure Can reason from system metrics back to model behavior: when latency, queueing, sampling, data order, rollout throughput ...

production ML systems Comfortable working across models, infrastructure and product Enjoy working in fast-moving, early-stage environments Tech Stack Python * PyTorch * JAX * LLMs * vLLM * Vector Databases * Modern Agent Frameworks Get in touch for more details - McGregor Boyall is an equal opportunity employer and do not discriminate on any grounds. ...

platform strategy. Strong understanding of AI software platforms, including inference, model deployment, runtimes, SDKs, developer tools, AI orchestration frameworks (e.g., Kubernetes‐based systems, Ray, vLLM, TGI), and cloud‐native and edge AI software stacks. Experience with data center AI infrastructure, including CPUs, GPUs, NPUs, heterogeneous accelerator environments, Kubernetes‐based environments ...

platform strategy. Strong understanding of AI software platforms, including: Inference, model deployment, runtimes, SDKs, developer tools AI orchestration frameworks (e.g., Kubernetes-based systems, Ray, vLLM, TGI) Cloud-native and edge AI software stacks Experience with data center AI infrastructure, including: CPUs, GPUs, NPUs, and heterogeneous accelerator environments Kubernetes-based environments ...

processing of high-volume inference requests Implement observability and cost-tracking for token usage across multiple LLM providers (Anthropic, OpenAI, open-source models via vLLM) Own database performance for both relational (Postgres) and vector (pgvector/Pinecone) workloads Collaborate with ML engineers on model-serving infrastructure and prompt-caching strategies … Node.js) Databases: PostgreSQL, Redis, Pinecone/pgvector Infra: AWS (ECS, Lambda, SQS/SNS), Docker, Kubernetes, Terraform AI/ML tooling: LangChain/LlamaIndex, vLLM, Anthropic & OpenAI APIs, embedding models Observability: Datadog, Grafana, OpenTelemetry CI/CD: GitHub Actions, ArgoCD Requirements: 4+ years backend development experience, ideally with at least ...

throughput and cost per token, partnering with the CUDA/GPU engineers. Make the core software architecture calls on serving frameworks and orchestration (e.g. vLLM, TensorRT-LLM, SGLang, Triton Inference Server, or equivalents). Translate throughput, latency, and uptime commitments into concrete technical specifications and serving capacity plans. ...

requests per second while maintaining sub‐second response times and cost efficiency. Experience with Golang is strongly preferred, and exposure to inference engines (vLLM, TGI, TensorRT), containerization, and distributed systems is an added bonus. You take ownership of platform‐level decisions, think strategically about performance vs. cost trade‐offs … experience building and scaling backend systems, distributed platforms, or inference infrastructure Strong understanding of AI/ML inference systems and experience with inference engines (vLLM, TGI, TensorRT‐LLM, or similar) Deep knowledge of distributed systems design, microservices architecture, and API gateway patterns Proficiency in Golang strongly preferred; Python, Rust, C++ ...

direction you shape across the platform. Responsibilities Own the Runtime & Serving Stack: Design, build, and extend the distributed inference and serving stack (e.g., vLLM, SGLang, NVIDIA Dynamo, TensorRT-LLM) onto DX-1, rather than treating any layer as a black box. Scale Distributed Inference: Define how inference scales across many … runtime/network/accelerator boundary. Demonstrated ownership of a hard, end-to-end systems problem, ideally extending a distributed inference/serving stack (vLLM, SGLang, NVIDIA Dynamo, TensorRT-LLM) in production, with specifics on what you built or changed and why. Distributed inference at scale: parallelism strategies, collective communication ...

direction you shape across the platform. Responsibilities Own the Runtime & Serving Stack: Design, build, and extend the distributed inference and serving stack (e.g. vLLM, SGLang, NVIDIA Dynamo, TensorRT-LLM) onto DX-1, rather than treating any layer as a black box. Scale Distributed Inference: Define how inference scales across many … runtime/network/accelerator boundary. Demonstrated ownership of a hard, end-to-end systems problem, ideally extending a distributed inference/serving stack (vLLM, SGLang, NVIDIA Dynamo, TensorRT-LLM) in production, with specifics on what you built or changed and why. Distributed inference at scale: parallelism strategies, collective communication ...