Open source alternatives to vLLM

One project on starwire does the same job. Ranked by how much of what vLLM does each one covers, read from its README and website. Ties go to the most active.

What vLLM does

Fast LLM inference servingcore
OpenAI-compatible API servercore
High throughput with batching
Support for many open models
Multi-GPU and quantized models
Structured output and tool calls

Each dot in the table is one job, in this order. Square dots are core jobs and count double. Full means yes, half means partly. "See the evidence" under a project shows the line behind each one. Coverage is what a project documents, not how well it works or how hard it is to move to. How we score this

Project	Covers	Stars	Commits, 12 weeks	Last release	Closed vs opened	Setup	License
FlideFast inference engine for LLM decision models	Same job, different way49%	505+385	63	2 days ago	93%	For developers	Apache-2.0
Covers 1 of 2 core jobs. Partly: Support for many open models; Multi-GPU and quantized models; Structured output and tool calls. Not shown: OpenAI-compatible API server.See the evidence Fast LLM inference servingcoreYesup to 9.5 times faster than vLLM and SGLang OpenAI-compatible API servercoreNot shownNeither the README nor the website mentions it. High throughput with batchingYes2.2 the throughput with 64 clients Support for many open modelsPartlyserving Qwen3.5 and Gemma 4 chat models Multi-GPU and quantized modelsPartlyQwen3.5 in FP8 on Ada, Hopper and Blackwell GPUs Structured output and tool callsPartlywith a probability for every option

Vital signs from GitHub, checked weekly. How we read these · How setup is read

All repos

Sort

Form Platform Works with Theme License Setup Language

New repos every two days, stars refreshed daily. Visits are counted without cookies, and the numbers are public. How it works Data RSS Newsletter