TensorFold
LLM Inference Engine for Metal, CUDA and Vulkan.
free(Free tier)1.4k github stars
About TensorFold
TensorFold 1.0.8 serves language models from a Zig binary on Apple Silicon and NVIDIA GPUs. The engine reads checkpoints, tokenizes requests and runs Metal or CUDA kernels directly. Serving needs no Python or MLX installation.