51 to 75 of 91 Permanent vLLM Jobs

semantic search solutions. The role requires hands-on experience with PyTorch, Hugging Face Transformers, and parameter-efficient fine-tuning, plus deploying models with vLLM, Ollama, or Triton. Strong software engineering practices are essential. #J-18808-Ljbffr ...

engineering & synthetic data pipelines Agentic frameworks, reasoning, tool use, memory systems Evaluation & benchmarking (LLM-as-judge, safety/alignment metrics) Inference optimisation (quantization, distillation, vLLM/TensorRT-LLM) Requirements: - PhD or Master's in CS/AI/ML/Physics/Maths or equivalent research experience - Strong hands ...

engineering & synthetic data pipelines Agentic frameworks, reasoning, tool use, memory systems Evaluation & benchmarking (LLM-as-judge, safety/alignment metrics) Inference optimisation (quantization, distillation, vLLM/TensorRT-LLM) Requirements: PhD or Master's in CS/AI/ML/Physics/Maths or equivalent research experience Strong hands ...

also highly valued: Post-graduate degrees and research experience in relevant fields (please list your publications). Deep understanding of inference serving frameworks (e.g. vLLM) Background in statistical analysis Contributions to open source and/or research projects Benefits A collaborative and supportive work environment The opportunity to have ...

ability to work both independently and as part of a team.Strong programming skills in Python and experience with machine learning libraries such as PyTorch, vLLM or similar are a prerequisiteYou will have, or be working towards gaining, a Masters or PhD degree in NLP or a related quantitative subject, such ...

SLOs, capacity planning, autoscaling strategies, and mentoring senior and mid‐level engineers. Preferred Direct open‐source or production contributions to modern inference frameworks (e.g., vLLM, Triton, TensorRT‐LLM, Ray Serve, or TorchServe). Deep experience with GPU systems engineering and hardware performance optimisation (e.g., CUDA, NCCL, RDMA, NUMA ...

product management in development tools and software including CLIs, SDKs and APIs.Knowledge of machine learning frameworks, runtimes, infrastructure such as PyTorch, ONNX, ExecuTorch, Llama.cpp, vLLM and LiteRT.Understanding of model deployment to edge, embedded or heterogeneous computing, with an understanding of trade-offs between accuracy, performance and power consumption.You should also ...

PyTorch in particular) Practical experience with model deployment workflows - loading, format conversion, quantisation, or framework integration Comfortable working with inference serving stacks (for example vLLM, TensorRT‐LLM, or similar) Familiarity with Linux, containerisation (Docker), and cluster environments Comfortable in a customer‐facing role, able to communicate clearly with ...

management in development tools and software including CLIs, SDKs and APIs. Knowledge of machine learning frameworks, runtimes, infrastructure such as PyTorch, ONNX, ExecuTorch, Llama.cpp, vLLM and LiteRT. Understanding of model deployment to edge, embedded or heterogeneous computing, with an understanding of trade-offs between accuracy, performance and power consumption. ...

testing Optimise reliability, scalability, latency, and user experience Work closely with Product and Engineering teams to solve complex customer problems Python PyTorch JAX vLLM Vector Databases What We're Looking For Strong software engineering fundamentals Experience building AI agents, copilots, RAG systems, or workflow automation platforms Excellent Python and/ ...

Computer Science, Electronic Engineering, Maths, Physics, or related degree and 5+ years of industry experience. You may also have: Experience with inference serving stacks (vLLM, SGLang, TensorRT-LLM, Triton) or ML compiler and runtime toolchains. Experience working with GPUs, TPUs, or other machine learning accelerators. A track record of contributing ...

compatible object storage - including performance-killing cases (small-object overhead, range-request patterns, eventual consistency, multipart tuning). Comfort with model-serving runtimes (vLLM, SGLang etc) and the formats they consume (safetensors, GGUF, sharded checkpoints). An end-to-end view: comfortable reasoning about NIC, switch, filesystem, cache, container runtime ...

training, evaluation, RLVR, PEFT, quantisation, tensor/data parallelism.Ideally, the candidate should have familiarity with CUDA kernels and/or Triton, and inference engines (vLLM, SGLang, et cetera).Experience coding with deep learning libraries such as Pytorch/JAX is essential.Fluent written and spoken English skills as well as contributions ...

reliability, performance, and cost-efficiency of the LLM inference platform end to end. You will operate large language model serving stacks (such as vLLM and llm-d) in production at scale, with deep instrumentation and strong operational rigor. You will partner across engineering to deliver secure software, improve stability … infrastructure Build backend services and APIs that enable reliable operation of AI infrastructure in production Operate and scale LLM serving infrastructure (such as vLLM and llm-d), including model hosting, request routing, continuous batching, and KV-cache optimization Deploy, host, and lifecycle-manage open-source and proprietary LLMs on Amazon ...

reliability, performance, and cost-efficiency of the LLM inference platform end to end. You will operate large language model serving stacks (such as vLLM and llm-d) in production at scale, with deep instrumentation and strong operational rigor. You will partner across engineering to deliver secure software, improve stability … infrastructure Build backend services and APIs that enable reliable operation of AI infrastructure in production Operate and scale LLM serving infrastructure (such as vLLM and llm-d), including model hosting, request routing, continuous batching, and KV-cache optimization Deploy, host, and lifecycle-manage open-source and proprietary LLMs on Amazon ...

reliability, performance, and cost-efficiency of the LLM inference platform end to end. You will operate large language model serving stacks (such as vLLM and llm-d) in production at scale, with deep instrumentation and strong operational rigor. You will partner across engineering to deliver secure software, improve stability … infrastructure Build backend services and APIs that enable reliable operation of AI infrastructure in production Operate and scale LLM serving infrastructure (such as vLLM and llm-d), including model hosting, request routing, continuous batching, and KV-cache optimization Deploy, host, and lifecycle-manage open-source and proprietary LLMs on Amazon ...

Strong knowledge of distributed systems, operating systems, machine learning systems architecture, Inference serving, and AI Infrastructure. Hands-on experience with LLM serving frameworks (e.g., vLLM, Ray Serve, TensorRT-LLM, TGI) and distributed KV cache optimization. Proficiency in C/C++, with additional experience in Python for research prototyping. Solid grounding ...

operating commonly used databases such as PostgreSQL, Clickhouse, and MongoDBPREFERRED SKILLS AND EXPERIENCE:Experience with LLM inference engines and serving frameworks (e.g., SGLang, TensorRT, vLLM)Experience designing or building with agent SDKs and agent orchestration frameworksExperience with Docker, Kubernetes, and containerized applicationsExpert knowledge of gRPC (unary, response streaming, bi-directional ...

preferably for LLM workloads. Proficiency in Python and PyTorch or JAX. Hands‐on experience with large‐scale LLM training or inference technologies, e.g., SGLang, vLLM, verl, Megatron, OpenRLHF. Good taste: you know when to build, when to buy, and when to delete. AI‐pilled: adopting agents, keen to build ...

infrastructure, ML systems, or computer architecture. Familiarity with Agile or other modern technical project management frameworks. Knowledge of modern inference‐serving frameworks (e.g., vLLM). Background in statistics, operations research, or large‐scale datacenter infrastructure. Contributions to open‐source AI or systems projects. Benefits High‐impact role in a rapidly ...

understanding of transformer inference, including prefill versus decode, KV-cache behaviour, attention variants and performance bottlenecksHands-on experience with LLM serving frameworks such as vLLM, SGLang, TensorRT-LLM or TGI, and the PyTorch ecosystemExperience with inference optimisation techniques, including quantisation, speculative decoding and model parallelism across modern GPU architecturesStrong software ...

transformer inference, including prefill versus decode, KV‐cache behaviour, attention variants and performance bottlenecks Hands‐on experience with LLM serving frameworks such as vLLM, SGLang, TensorRT‐LLM or TGI, and the PyTorch ecosystem Experience with inference optimisation techniques, including quantisation, speculative decoding and model parallelism across modern GPU architectures Strong ...

guide peers on safe and effective usage within team practices Preferred qualifications, capabilities, and skills Experience operating large language model inference servers such as vLLM and llm-d or directly equivalent model serving stacks in production Experience developing generative AI applications, AI agents, vector search, and retrieval-augmented generation patterns … monitoring, including hallucination detection, toxicity filtering, and drift detection using open telemetry conventions Contributions to open-source large language model serving or inference projects, (vLLM, llm-d, Ray, KServe, Triton) #J-18808-Ljbffr ...

guide peers on safe and effective usage within team practices Preferred qualifications, capabilities, and skills Experience operating large language model inference servers such as vLLM and llm-d (or directly equivalent model serving stacks) in production Experience developing generative AI applications, AI agents, vector search, and retrieval-augmented generation patterns … monitoring, including hallucination detection, toxicity filtering, and drift detection using open telemetry conventions Contributions to open-source large language model serving or inference projects, (vLLM, llm-d, Ray, KServe, Triton) ABOUT US J.P. Morgan is a global leader in financial services, providing strategic advice and products to the world ...

ability to guide peers on safe and effective usage within team practicesPreferred qualifications, capabilities, and skillsExperience operating large language model inference servers such as vLLM and llm-d (or directly equivalent model serving stacks) in productionExperience developing generative AI applications, AI agents, vector search, and retrieval-augmented generation patternsExperience building … quality monitoring, including hallucination detection, toxicity filtering, and drift detection using open telemetry conventionsContributions to open-source large language model serving or inference projects, (vLLM, llm-d, Ray, KServe, Triton)J.P. Morgan is a global leader in financial services, providing strategic advice and products to the world’s most prominent ...