TensorRT-LLMhttps://github.com/NVIDIA/TensorRT-LLMNVIDIA's optimized LLM inference. ExLlamaV2https://github.com/turboderp-org/exllamav2Fast local inference kernels. Modalhttps://modal.comServerless GPU/CPU compute. Novita AIhttps://novita.aiServerless GPU model APIs. vLLMhttps://github.com/vllm-project/vllmHigh-throughput GPU serving for open models.