Ollamahttps://ollama.comThe default local model runner; CLI + OpenAI-compatible API. ComfyUIhttps://www.comfy.orgNode-graph image/video generation UI. llama.cpphttps://github.com/ggml-org/llama.cppGGUF inference engine underneath most local runners. vLLMhttps://github.com/vllm-project/vllmHigh-throughput GPU serving for open models. whisper.cpphttps://github.com/ggerganov/whisper.cppLocal Whisper inference. LocalAIhttps://localai.ioSelf-hosted OpenAI-compatible API. text-generation-webuihttps://github.com/oobabooga/text-generation-webuiFeature-rich local web UI for models. exohttps://github.com/exo-explore/exoCluster everyday devices into a GPU pool. Janhttps://jan.aiOpen-source local AI desktop app. AirLLMhttps://github.com/lyogavin/airllmRun 70B models on 4GB RAM via layer-wise inference. Tabbyhttps://tabby.tabbyml.comSelf-hosted coding assistant. InvokeAIhttps://invoke-ai.github.ioVisual creative engine for diffusion. MLC LLMhttps://llm.mlc.aiCompile models for every device incl. phones/browser. KoboldCpphttps://github.com/LostRuins/koboldcppSingle-binary local runner. Text Generation Inferencehttps://github.com/huggingface/text-generation-inferenceHF's production serving stack. Hyperspace Podshttps://hyperspaceai.agiDecentralized inference pods network. LM Studiohttps://lmstudio.aiPolished local model desktop app + local server. Colibri.aihttps://github.com/nickr-s/ColibriLightweight local inference toolkit.