Index of every page on this site: llms.txt
All skills
vllm-project avatar

vllm-project/vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

main Updated 5 days agoGitHub
README badge for vllm-project/vllm

Repository statistics

  • Indexed skills

    8

  • Skill groups

    1

  • GitHub stars

    97,041

  • Forks

    24,900

8 total

Prepare a vLLM change for human PR review or re-review, or analyze an open pull request. Use before opening a PR, drafts included (e.g. `gh pr create`), or pushing review fixes, to check design fit, behavioral coverage, performance evidence, diff quality, and closure of previous feedback.

Updated

Find vLLM deployment recipes and configs for LLMs and Hugging Face models by model and hardware. Use for recipe lookup, creation, and updates.

Updated

Debug CUDA illegal memory access (IMA) errors in vLLM with CUDA core dumps and cuda-gdb. Use when a vLLM run crashes or hangs with a CUDA illegal memory access, device-side assert, or other asynchronous CUDA error and the faulting kernel needs to be located.

Updated

Build, debug, and interpret vLLM GPU kernel microbenchmarks for CUDA, ROCm/HIP, Triton, and CuteDSL, including CUPTI timing, correctness checks, generated-code inspection, multi-GPU measurements, and SOL sanity checks. Use when measuring or comparing the performance of a single GPU kernel or op, validating a claimed kernel speedup, or investigating a noisy or surprising benchmark result.

Updated

Write or review Triton kernels for vLLM, with practical guidance for generated-code inspection, launch grids, indexing, specialization, tuning, and representative performance validation. Use when adding, modifying, optimizing, or reviewing a `@triton.jit` kernel in vLLM.

Updated

Playbook for changing vLLM's Transformers version bounds. Use when bumping the pinned or maximum Transformers version, raising the minimum Transformers version, or deleting vendored configs, processors, or compat shims that upstream Transformers now provides.

Updated

Determine the correct behavior of vLLM's OpenAI-compatible or Anthropic-compatible API by checking the official upstream spec, instead of guessing from vLLM's own code. Use for any bug, issue, PR, review, or question touching /v1/chat/completions, /v1/completions, /v1/responses, /v1/embeddings, /v1/messages, or their request/response models — including "is this a bug or intended", "what should this field default to", "does upstream allow this", and any change that adds, removes, or renames a field on those models. Do not use for vLLM-internal APIs (engine, scheduler, LLM class) or for endpoints with no upstream counterpart.

Updated

Fetch and diagnose vLLM Buildkite CI failure logs. Use when investigating failing CI jobs on a PR or build, when the user pastes a buildkite.com URL, or asks to fetch/diagnose CI logs.

Updated

The badge links readers to this page. It shows the skilld mark and no counts, and it follows the reader's light or dark GitHub theme.

<a href="https://skilld.dev/gh/vllm-project/vllm"> <picture> <source media="(prefers-color-scheme: dark)" srcset="https://skilld.dev/b/vllm-project/vllm?theme=dark"> <source media="(prefers-color-scheme: light)" srcset="https://skilld.dev/b/vllm-project/vllm?theme=light"> <img alt="Skill repository on skilld.dev" src="https://skilld.dev/b/vllm-project/vllm?theme=light"> </picture> </a>