vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
free（無料プランあり）94k github stars
vllm について
A high-throughput and memory-efficient inference and serving engine for LLMs
A high-throughput and memory-efficient inference and serving engine for LLMs
A high-throughput and memory-efficient inference and serving engine for LLMs