51 to 75 of 75 vLLM Jobs in the UK

product management in development tools and software including CLIs, SDKs and APIs.Knowledge of machine learning frameworks, runtimes, infrastructure such as PyTorch, ONNX, ExecuTorch, Llama.cpp, vLLM and LiteRT.Understanding of model deployment to edge, embedded or heterogeneous computing, with an understanding of trade-offs between accuracy, performance and power consumption.You should also ...

PyTorch in particular) Practical experience with model deployment workflows - loading, format conversion, quantisation, or framework integration Comfortable working with inference serving stacks (for example vLLM, TensorRT‐LLM, or similar) Familiarity with Linux, containerisation (Docker), and cluster environments Comfortable in a customer‐facing role, able to communicate clearly with ...

management in development tools and software including CLIs, SDKs and APIs. Knowledge of machine learning frameworks, runtimes, infrastructure such as PyTorch, ONNX, ExecuTorch, Llama.cpp, vLLM and LiteRT. Understanding of model deployment to edge, embedded or heterogeneous computing, with an understanding of trade-offs between accuracy, performance and power consumption. ...

reliability, performance, and cost-efficiency of the LLM inference platform end to end. You will operate large language model serving stacks (such as vLLM and llm-d) in production at scale, with deep instrumentation and strong operational rigor. You will partner across engineering to deliver secure software, improve stability … infrastructure Build backend services and APIs that enable reliable operation of AI infrastructure in production Operate and scale LLM serving infrastructure (such as vLLM and llm-d), including model hosting, request routing, continuous batching, and KV-cache optimization Deploy, host, and lifecycle-manage open-source and proprietary LLMs on Amazon ...

Strong knowledge of distributed systems, operating systems, machine learning systems architecture, Inference serving, and AI Infrastructure. Hands-on experience with LLM serving frameworks (e.g., vLLM, Ray Serve, TensorRT-LLM, TGI) and distributed KV cache optimization. Proficiency in C/C++, with additional experience in Python for research prototyping. Solid grounding ...

operating commonly used databases such as PostgreSQL, Clickhouse, and MongoDBPREFERRED SKILLS AND EXPERIENCE:Experience with LLM inference engines and serving frameworks (e.g., SGLang, TensorRT, vLLM)Experience designing or building with agent SDKs and agent orchestration frameworksExperience with Docker, Kubernetes, and containerized applicationsExpert knowledge of gRPC (unary, response streaming, bi-directional ...

guide peers on safe and effective usage within team practices Preferred qualifications, capabilities, and skills Experience operating large language model inference servers such as vLLM and llm-d or directly equivalent model serving stacks in production Experience developing generative AI applications, AI agents, vector search, and retrieval-augmented generation patterns … monitoring, including hallucination detection, toxicity filtering, and drift detection using open telemetry conventions Contributions to open-source large language model serving or inference projects, (vLLM, llm-d, Ray, KServe, Triton) #J-18808-Ljbffr ...

guide peers on safe and effective usage within team practices Preferred qualifications, capabilities, and skills Experience operating large language model inference servers such as vLLM and llm-d (or directly equivalent model serving stacks) in production Experience developing generative AI applications, AI agents, vector search, and retrieval-augmented generation patterns … monitoring, including hallucination detection, toxicity filtering, and drift detection using open telemetry conventions Contributions to open-source large language model serving or inference projects, (vLLM, llm-d, Ray, KServe, Triton) ABOUT US J.P. Morgan is a global leader in financial services, providing strategic advice and products to the world ...

ability to guide peers on safe and effective usage within team practicesPreferred qualifications, capabilities, and skillsExperience operating large language model inference servers such as vLLM and llm-d (or directly equivalent model serving stacks) in productionExperience developing generative AI applications, AI agents, vector search, and retrieval-augmented generation patternsExperience building … quality monitoring, including hallucination detection, toxicity filtering, and drift detection using open telemetry conventionsContributions to open-source large language model serving or inference projects, (vLLM, llm-d, Ray, KServe, Triton)J.P. Morgan is a global leader in financial services, providing strategic advice and products to the world’s most prominent ...

preferably for LLM workloads. Proficiency in Python and PyTorch or JAX. Hands‐on experience with large‐scale LLM training or inference technologies, e.g., SGLang, vLLM, verl, Megatron, OpenRLHF. Good taste: you know when to build, when to buy, and when to delete. AI‐pilled: adopting agents, keen to build ...

infrastructure, ML systems, or computer architecture. Familiarity with Agile or other modern technical project management frameworks. Knowledge of modern inference‐serving frameworks (e.g., vLLM). Background in statistics, operations research, or large‐scale datacenter infrastructure. Contributions to open‐source AI or systems projects. Benefits High‐impact role in a rapidly ...

understanding of transformer inference, including prefill versus decode, KV-cache behaviour, attention variants and performance bottlenecksHands-on experience with LLM serving frameworks such as vLLM, SGLang, TensorRT-LLM or TGI, and the PyTorch ecosystemExperience with inference optimisation techniques, including quantisation, speculative decoding and model parallelism across modern GPU architecturesStrong software ...

user and a working results. Bonus Points These aren't requirements, but they'd make you stand out: Experience with inference serving stacks (vLLM, SGLang, TensorRT-LLM, Triton). A track record of contributing to or maintaining open source developer tools and ML ecosystem projects Experience writing technical documentation ...

improve throughput and cost per token, partnering with the CUDA/GPU engineersMake the core software architecture calls on serving frameworks and orchestration (e.g. vLLM, TensorRT-LLM, SGLang, Triton Inference Server or equivalents)Translate throughput, latency and uptime commitments into concrete technical specifications and serving capacity plansAct as direct technical ...

transformer inference, including prefill versus decode, KV‐cache behaviour, attention variants and performance bottlenecks Hands‐on experience with LLM serving frameworks such as vLLM, SGLang, TensorRT‐LLM or TGI, and the PyTorch ecosystem Experience with inference optimisation techniques, including quantisation, speculative decoding and model parallelism across modern GPU architectures Strong ...

throughput and cost per token, partnering with the CUDA/GPU engineers. Make the core software architecture calls on serving frameworks and orchestration (e.g. vLLM, TensorRT-LLM, SGLang, Triton Inference Server, or equivalents). Translate throughput, latency, and uptime commitments into concrete technical specifications and serving capacity plans. ...

optimized in a Data Center AI software platforms Inference, model deployment, runtimes, SDKs, developer tools. AI orchestration frameworks, e.g., Kubernetes-based systems, Ray, vLLM, TGI. Cloud-native AI software stacks, deployment patterns, and data center operating models. Data center AI infrastructure CPUs, GPUs, NPUs, and heterogeneous accelerator environments. Kubernetes-based ...

Opportunity We are seeking a Senior Software Engineer with a passion for open-source AI infrastructure to work on deploying, extending and optimising vLLM (and potentially other inference serving engines) to support our projects. You will play a crucial, high-impact role across both our key programmes, the Scaling Inference … autonomous agents for software development. What You'll Do Deploy, instrument and monitor open weight models served using vLLM. Implement new features within vLLM to support novel hardware architectures as part of the Scaling Inference Lab. Work with the Panopticon team to identify opportunities to extend vLLM to enhance accuracy ...

orchestration framework (LangGraph, LangChain/LlamaIndex, CrewAI, or AutoGen), and vector databases (Pinecone, Qdrant, Weaviate, or Milvus) Model‐serving infrastructure: For example Triton/vLLM as well as LLM gateway products such as LiteLLM or Bifrost A production track record: Designing complex RAG pipelines, fine‐tuning foundation models (LoRA/… continuous evaluation pipelines, guardrails, observability, and cost‐optimization strategy for production LLM deployments. Decide the stack: Own build‐vs‐buy calls e.g. self‐hosted vLLM vs. a managed endpoint and the RAG‐vs‐fine‐tune tradeoffs that other squads calibrate against. Lead technical culture: Set technical direction across AI engineering ...

results focus. What you will do: Operate the LLM estate: run LoRA fine-tuning cycles and evaluation gates on our GPU hardware, manage vLLM serving (including multi-adapter deployments) alongside production, promote or roll back model versions on the gate results, and keep the serving watchdogs healthy. The CEO retains … world system of record. Python for scripting, data processing or pipeline work. Preferable: Deeper LLM experience: fine-tuning methodology, corpus design, evaluation-harness construction, vLLM internals, or heavy daily use of AI coding agents. ERP, FSM or works-management domain experience job lifecycles, scheduling, SLAs, parts, timesheets, invoicing or heat ...

placement, quota management, and utilization systems across heterogeneous accelerator environments.Optimize inference pipelines for latency, throughput, reliability, memory efficiency, and cost using frameworks such as vLLM, Triton Inference Server, TensorRT, or equivalent serving stacks.Partner with ML engineers and researchers to remove bottlenecks in training, evaluation, batch inference, online inference, deployment … monitoring, and cost optimization.Strong systems engineering fundamentals across Linux, networking, storage, containers, Kubernetes, distributed runtimes, and production debugging.Experience with ML serving frameworks such as vLLM, Triton Inference Server, TensorRT, TorchServe, KServe, Ray Serve, or equivalent systems.Proficiency in Python for infrastructure automation, tooling, debugging, integration, and operational workflows.Practical understanding of performance ...

quota management, and utilization systems across heterogeneous accelerator environments. Optimize inference pipelines for latency, throughput, reliability, memory efficiency, and cost using frameworks such as vLLM, Triton Inference Server, TensorRT, or equivalent serving stacks. Partner with ML engineers and researchers to remove bottlenecks in training, evaluation, batch inference, online inference, deployment … cost optimization. Strong systems engineering fundamentals across Linux, networking, storage, containers, Kubernetes, distributed runtimes, and production debugging. Experience with ML serving frameworks such as vLLM, Triton Inference Server, TensorRT, TorchServe, KServe, Ray Serve, or equivalent systems. Proficiency in Python for infrastructure automation, tooling, debugging, integration, and operational workflows. Practical understanding ...

CommonAI CIC is seeking a Senior Software Engineer to help deploy, extend and optimise vLLM and related inference engines for our AI infrastructure projects. You will work across the Scaling Inference Lab and the High Assurance programme to drive performance, reliability and safety in production‐grade systems. You will collaborate ...

depth profiling of large-scale inference pipelines, specifically focusing on KV cache management and heterogeneous memory scheduling. AI Serving: Optimising high-throughput frameworks (vLLM, Ray Serve, PyTorch Distributed) to ensure low-latency, multi-tenant performance. Research Leadership: Contributing to top-tier venues (OSDI, NSDI, EuroSys, MLSys) and driving those innovations … Stack: Strong proficiency in C/C++ for systems work, with Python for rapid prototyping. Expertise: Hands-on experience with LLM serving frameworks ( vLLM, Ray Serve, TensorRT-LLM ) and distributed algorithms. Mindset: A solid grounding in systems research methodology and performance profiling tools. The "Value Add" (Desired): A PhD focused ...

direction you shape across the platform. Responsibilities Own the Runtime & Serving Stack: Design, build, and extend the distributed inference and serving stack (e.g. vLLM, SGLang, NVIDIA Dynamo, TensorRT-LLM) onto DX-1, rather than treating any layer as a black box. Scale Distributed Inference: Define how inference scales across many … runtime/network/accelerator boundary. Demonstrated ownership of a hard, end-to-end systems problem, ideally extending a distributed inference/serving stack (vLLM, SGLang, NVIDIA Dynamo, TensorRT-LLM) in production, with specifics on what you built or changed and why. Distributed inference at scale: parallelism strategies, collective communication ...