TensorFold
LLM Inference Engine for Metal, CUDA and Vulkan.
free（有免费版）1.4k github stars
TensorFold 简介
TensorFold 1.0.8 serves language models from a Zig binary on Apple Silicon and NVIDIA GPUs. The engine reads checkpoints, tokenizes requests and runs Metal or CUDA kernels directly. Serving needs no Python or MLX installation.