Skip to main content

KServe

还没翻译译自英文(docs.vllm.ai)本页 3 段都还没译

KServe

vLLM can be deployed with KServe on Kubernetes for highly scalable distributed model serving.

You can use vLLM with KServe's Hugging Face serving runtime or via LLMInferenceService that uses llm-d.