TensorFold
LLM Inference Engine for Metal, CUDA and Vulkan.
free（無料プランあり）1.4k github stars
TensorFold について
TensorFold 1.0.8 serves language models from a Zig binary on Apple Silicon and NVIDIA GPUs. The engine reads checkpoints, tokenizes requests and runs Metal or CUDA kernels directly. Serving needs no Python or MLX installation.