- llama.cpp by ggml-org, 135k stars. LLM inference in C and C++
- vllm by vllm-project, 97k stars. High-throughput inference and serving engine for large language models
- unsloth by unslothai, 80k stars. Local UI for running and fine-tuning LLMs and diffusion models
- voicebox by jamiepine, 61k stars. Voice studio for cloning voices, dictation and generating speech
- colibri by JustVugg, 51k stars. Inference engine for MoE models that spreads experts across GPU, RAM and NVMe
- Strata by Niko1221, 44k stars. Local inference engine for Qwen models with an OpenAI-compatible localhost API
- project-nomad by Crosstalk-Solutions, 44k stars. Offline knowledge server with Wikipedia, books, courses, maps and optional local AI
- airllm by lyogavin, 42k stars. Runs large language models layer by layer so they fit in a small GPU's memory
- BitNet by microsoft, 42k stars. Inference framework for running 1-bit quantized large language models
- llmfit by AlexsJones, 39k stars. CLI that reports which local LLMs and providers fit your hardware
- ds4 by antirez, 24k stars. Local inference engine for DeepSeek models on Metal, CUDA and ROCm
- omlx by jundot, 24k stars. LLM inference server for Apple Silicon with batching, SSD cache and OpenAI API
- ktransformers by kvcache-ai, 20k stars. Framework for LLM inference and fine-tuning on mixed hardware
- web-llm by mlc-ai, 19k stars. In-browser LLM inference engine using WebGPU
- FreeToken by FlashML-org, 17k stars. Local model server for running large MoE models on a desktop
- OpenLLM by bentoml, 13k stars. Serve open-source LLMs like Llama and DeepSeek behind an OpenAI-compatible API
- kimi-k3-in-c by FareedKhan-dev, 12k stars. Kimi K3 inference on a single CPU in portable C99 with no dependencies
- Soup by MakazhanAlpamys, 11k stars. Fine-tunes LLMs from a YAML config, streaming layers to fit small GPUs
- modly by lightningpixel, 8.7k stars. Desktop app that makes 3D models from images or text prompts on your GPU
- osaurus by osaurus-ai, 8.2k stars. Offline macOS app for running agents with local models and persistent memory
- ODS by Osmantic, 7.8k stars. Pre-release stack that turns a PC or Mac into a private AI server with Docker
- magnitude by magnitudedev, 7.8k stars. Inference engine that tunes kernels per hardware for faster model runs
- turbo-fieldfare by drumih, 7.4k stars. Runs a Gemma model on Apple Silicon Macs with low memory use via Metal
- whichllm by Andyyyy64, 6.8k stars. CLI that ranks local LLMs by benchmark results for your hardware
- lemonade by lemonade-sdk, 6.2k stars. Local server that runs optimized LLMs on AMD GPUs and NPUs
- transformerlab-app by transformerlab, 5.2k stars. Desktop app to train, evaluate and scale models from laptop to GPU cluster
- locally-uncensored by PurpleDoubleD, 4.3k stars. Desktop app for local chat, image and video generation, and a coding agent
- mesh-llm by Mesh-LLM, 4.1k stars. Pools private or public compute to run distributed LLM inference for agents
- atomic-agent by AtomicBot-ai, 3.7k stars. Local AI agent that runs open-weight GGUF models through llama.cpp
- Bonsai-demo by PrismML-Eng, 3.5k stars. Demo of small Bonsai language models running with llama.cpp and MLX
- claude-code-local by nicedreamzapp, 3.5k stars. Runs Claude Code against local models on Mac, Linux and Windows with no API key
- OGAM by off-grid-ai, 3.4k stars. Offline app for running GGUF models, vision, speech, and image generation locally
- ramalama by containers, 3.1k stars. CLI that serves AI models locally in containers for inference
- FreeVideo by FlashML-org, 3.1k stars. Local video generation with MiniMax H3, tuned for 8 GB VRAM cards
- h3.c by antirez, 3.1k stars. Inference engine for a MiniMax model, targeting Mac computers in C
- lucebox by Luce-Org, 3.0k stars. Inference server that speeds up local LLMs on consumer GPUs
- tokenspeed by lightseekorg, 2.9k stars. Inference engine for serving LLMs with a focus on speed
- HyperQwen by syv-ai, 2.7k stars. vLLM setup and quantization tuning for serving Qwen models on a single GPU
- nanocoder by Nano-Collective, 2.6k stars. Open-source terminal coding agent that works with any model you bring
- warp by sqliteai, 2.6k stars. Dependency free C runtime that streams large model weights from NVMe
New repos every two days, stars refreshed daily. Visits are counted without cookies, and the numbers are public. How it works Data RSS Newsletter