vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
free（有免费版）94k github stars
vllm 简介
A high-throughput and memory-efficient inference and serving engine for LLMs
A high-throughput and memory-efficient inference and serving engine for LLMs
A high-throughput and memory-efficient inference and serving engine for LLMs