- cua by trycua, 32k stars. Open-source drivers and cross-OS sandboxes for computer-use agents
- deepeval by confident-ai, 19k stars. Python framework with metrics for evaluating LLM outputs
- DeepSpec by deepseek-ai, 7.3k stars. Training and evaluation code for speculative decoding algorithms
- whichllm by Andyyyy64, 6.8k stars. CLI that ranks local LLMs by benchmark results for your hardware
- awesome-harness-engineering by ai-boost, 5.2k stars. Curated list of tools and patterns for building agent harnesses
- inspector by MCPJam, 2.3k stars. Web app to chat with, test and debug MCP servers and ChatGPT apps
- fable-method by Sahir619, 2.3k stars. Workflow skills for coding agents, with an eval to check the results
- nimble by bespokelabsai, 2.3k stars. Python toolkit for typed local decisions, contrastive data curation, and model evals
- future-agi by future-agi, 2.3k stars. Self-hostable platform for tracing, evaluating and guarding LLM and agent apps
- livenerf by ninjahawk, 1.9k stars. Benchmark that tracks how a model's capability changes after release
- frontier-evals by openai, 1.3k stars. Evaluation harness for testing frontier model capabilities
- ClickBench by ClickHouse, 1.2k stars. Benchmark suite that compares query performance of analytical databases
- ProgramBench by facebookresearch, 980 stars. Benchmark testing whether language models can rebuild programs from scratch
- agent-skills-eval by darkrishabh, 817 stars. Test runner for agent skills, with JSONL-based eval cases
- overmind by overmind-core, 759 stars. Platform for evaluating and fine-tuning LLM agents over time
- ai-engineer-notebooks by calmrocks, 749 stars. Framework-free Colab notebooks on model APIs, RAG, evals and agents
- SkillCorpus by EverMind-AI, 703 stars. Builds curated, searchable corpora from SKILL.md files with evaluation tools
- ParseBench by run-llama, 638 stars. Benchmark that scores document parsers on tasks aimed at AI agents
- embodied-first-exam by mll-lab-nu, 276 stars. Benchmark where code agents solve robot tasks with programs run in a sandbox
- civ6-mcp by lmwilki, 241 stars. MCP server letting an LLM agent play Civilization VI turn by turn
- RSI-Exam by aiming-lab, 198 stars. Benchmark for recursive self-improvement on long executable research tasks
- autolab by autolabhq, 172 stars. Benchmark for testing AI agents on long-horizon automated research tasks
- benchmark by browser-use, 171 stars. Benchmark suite of 200 real web tasks for evaluating browser agents
- XRepoTest by solis-team, 168 stars. Benchmark for LLM unit test generation across five languages in Docker
- LoopArena by AMAP-ML, 155 stars. Benchmark that tests models as controllers for coding agent loops
- lora-speedrun by Saivineeth147, 151 stars. Public wall-clock leaderboard for LoRA fine-tuning on a fixed task
- adlc-team-skills by tikalk, 145 stars. Agent skills for team lifecycle, evals and architecture decision records
- jevals by openlayer-ai, 120 stars. Evals and guardrails for agent traces, run locally with Kev or Laya
- aistupidmeter-web by StudioPlatforms, 113 stars. Web app showing continuous LLM benchmarks, drift detection and routing
- diffusion-bench by End2End-Diffusion, 109 stars. Benchmark suite for evaluating diffusion transformer generative models
- deltafin by argonautlabsai, 100 stars. Streams a 2.8T MoE model from SSD on Apple Silicon, with benchmarks
- DishwasherBench by composo-ai, 74 stars. Browser game on a physical dishwasher simulation used to benchmark language models
- DAS by ZhikaiXu24, 67 stars. Dataset and benchmark for generating academic surveys with LLMs
- pi-bench by kyuz0, 66 stars. Benchmark runner that tests pi-coding-agent against real repository tasks
- cupel by tolitius, 65 stars. Finds LLMs that perform better than their size suggests, for local use
- homebench by david-g-3654, 59 stars. CLI that benchmarks local models for speed, memory, and quality with a TUI
- Qwen3.8-27B-FP8-on-a-single-DGX-Spark by 0xBakeer, 56 stars. Benchmarks and setup notes for running a 27B FP8 Qwen model on one DGX Spark
- medical-STT-eval by Omi-Health, 54 stars. Benchmark code for speech-to-text models on medical conversations
- research-chatgpt-guesses-between-1-and-100 by exmergo, 47 stars. Experiment showing ChatGPT's answers to random 1-100 picks are not uniform
- AutoSciRub by zjunlp, 43 stars. Automatic rubric induction to evaluate autonomous research agents
New repos every two days, stars refreshed daily. Visits are counted without cookies, and the numbers are public. How it works Data RSS Newsletter