- ollama by ollama, 187k stars. Run open-weight language models locally from the command line and an API
- llama.cpp by ggml-org, 135k stars. LLM inference in C and C++
- vllm by vllm-project, 97k stars. High-throughput inference and serving engine for large language models
- unsloth by unslothai, 80k stars. Local UI for running and fine-tuning LLMs and diffusion models
- whisper.cpp by ggml-org, 55k stars. C and C++ port of OpenAI's Whisper speech recognition model
- colibri by JustVugg, 51k stars. Inference engine for MoE models that spreads experts across GPU, RAM and NVMe
- jan by janhq, 46k stars. Desktop app for running open LLMs locally as a ChatGPT alternative
- Strata by Niko1221, 44k stars. Local inference engine for Qwen models with an OpenAI-compatible localhost API
- airllm by lyogavin, 42k stars. Runs large language models layer by layer so they fit in a small GPU's memory
- BitNet by microsoft, 42k stars. Inference framework for running 1-bit quantized large language models
- llmfit by AlexsJones, 39k stars. CLI that reports which local LLMs and providers fit your hardware
- DeepInfer by deepseek-ai, 26k stars. Inference framework for serving DeepSeek models at scale
- ds4 by antirez, 24k stars. Local inference engine for DeepSeek models on Metal, CUDA and ROCm
- omlx by jundot, 24k stars. LLM inference server for Apple Silicon with batching, SSD cache and OpenAI API
- ktransformers by kvcache-ai, 20k stars. Framework for LLM inference and fine-tuning on mixed hardware
- web-llm by mlc-ai, 19k stars. In-browser LLM inference engine using WebGPU
- FreeToken by FlashML-org, 17k stars. Local model server for running large MoE models on a desktop
- transformers.js by huggingface, 16k stars. Run Hugging Face transformer models in the browser with no server
- TensorRT-LLM by NVIDIA, 15k stars. Python API and runtime for fast LLM inference on NVIDIA GPUs
- cactus by cactus-compute, 15k stars. Small on-device model for agentic, voice and vision tasks on consumer devices
- kimi-k3-in-c by FareedKhan-dev, 12k stars. Kimi K3 inference on a single CPU in portable C99 with no dependencies
- Edge0 by Edge0-AI, 12k stars. Runs large mixture-of-experts models with weights streamed from SSD
- WhisperLiveKit by QuentinFuxa, 11k stars. Local streaming speech-to-text with speaker diarization and translation
- Soup by MakazhanAlpamys, 11k stars. Fine-tunes LLMs from a YAML config, streaming layers to fit small GPUs
- ODS by Osmantic, 7.8k stars. Pre-release stack that turns a PC or Mac into a private AI server with Docker
- magnitude by magnitudedev, 7.8k stars. Inference engine that tunes kernels per hardware for faster model runs
- turbo-fieldfare by drumih, 7.4k stars. Runs a Gemma model on Apple Silicon Macs with low memory use via Metal
- whichllm by Andyyyy64, 6.8k stars. CLI that ranks local LLMs by benchmark results for your hardware
- lemonade by lemonade-sdk, 6.2k stars. Local server that runs optimized LLMs on AMD GPUs and NPUs
- gpu-perf-engineering-resources by wafer-ai, 4.4k stars. Curated reading list on GPU fundamentals and production inference performance
- zml by zml, 4.3k stars. Inference stack in Zig for running models on any hardware
- mesh-llm by Mesh-LLM, 4.1k stars. Pools private or public compute to run distributed LLM inference for agents
- PhiCookBook by microsoft, 3.9k stars. Getting-started notebooks and guides for Microsoft's Phi small language models
- Bonsai-demo by PrismML-Eng, 3.5k stars. Demo of small Bonsai language models running with llama.cpp and MLX
- claude-code-local by nicedreamzapp, 3.5k stars. Runs Claude Code against local models on Mac, Linux and Windows with no API key
- OGAM by off-grid-ai, 3.4k stars. Offline app for running GGUF models, vision, speech, and image generation locally
- ramalama by containers, 3.1k stars. CLI that serves AI models locally in containers for inference
- h3.c by antirez, 3.1k stars. Inference engine for a MiniMax model, targeting Mac computers in C
- lucebox by Luce-Org, 3.0k stars. Inference server that speeds up local LLMs on consumer GPUs
- tokenspeed by lightseekorg, 2.9k stars. Inference engine for serving LLMs with a focus on speed
New repos every two days, stars refreshed daily. Visits are counted without cookies, and the numbers are public. How it works Data RSS Newsletter