Every entry answers the four questions that actually matter: what is it, what is it good at, can this machine run it, and can you fine-tune it here.
Catalogue figures come from each model’s published card. ModelLM has not independently measured them.
Microsoft
Strong reasoning at 3.8B with a 128k context — but full multi-head attention makes its KV cache expensive.
Small enough for a laptop CPU or a 6 GB card, and still coherent. A sensible target for edge deployment.
Hugging Face
Fully open training data and recipe. A good base when you want to understand exactly what the model saw.
A genuinely tiny model for classification, routing and extraction rather than open-ended chat.
BAAI
Multilingual retrieval across 100+ languages, with dense, sparse and multi-vector output from one model.
Nomic AI
The default local embedding model for Knowledge mode. 8k context and Matryoshka truncation, so you can trade index size for accuracy.
Alibaba Qwen
Frontier-adjacent open weights. Needs a workstation, a multi-GPU rig or a large unified-memory Mac.
Delivers close to Llama 3.1 405B quality at a size a dual-GPU workstation or 64 GB Mac can actually hold.
Mistral AI
The original open mixture-of-experts. 47B resident, ~13B active — fast generation if you have the memory to hold it.
Alibaba Qwen
Approaches 70B quality at half the memory. Runs on a single 24 GB card at Q4_K_M with a modest context window.
Alibaba Qwen
The strongest open code model that still fits one 24 GB card. The reason a lot of people buy a 4090.
DeepSeek
The strongest open reasoning model that fits a single 24 GB card. A genuinely different capability class on hard problems.
Alibaba Qwen
A mixture-of-experts model: 30B of weights in memory but only ~3B active per token, so it generates at roughly small-model speed.
The largest Gemma 3. Multimodal, long-context, and designed to run on a single high-memory accelerator.
Mistral AI
Built deliberately for low latency on a single card — fewer layers, wider FFN. Apache 2.0 at a size that usually is not.
BigCode
A base completion model trained on permissively-licensed source with full provenance. Not a chat model.
DeepSeek
A 16B MoE code model with only 2.4B active per token — fast enough for inline completion while holding 128k of context.
Alibaba Qwen
The sweet spot for 24 GB cards. Meaningfully stronger reasoning than 7B while still fine-tunable locally with QLoRA.
Alibaba Qwen
The Qwen3 mid-size. Long context and a reasoning mode inside a footprint a 24 GB card handles comfortably.
DeepSeek
Qwen2.5 14B distilled on R1 reasoning traces. Writes out its thinking before answering, which costs tokens but wins on hard problems.
Microsoft
Trained largely on curated synthetic data. Punches far above its size on reasoning and maths; the short context limits what you can do with it.
Mistral AI / NVIDIA
A 12B with a 128k context and the Tekken tokenizer, which compresses non-English text far better than Llama’s.
Multimodal and long-context in a 12B footprint. Interleaved local/global attention keeps the KV cache affordable.
Unusually good at natural, well-structured prose for its size. The short context is the catch.
Alibaba Qwen
Switchable thinking mode: the same weights answer directly or reason step by step depending on the prompt. Long context for its size.
The most widely supported open model there is. If a tool, adapter or tutorial exists, it was written for this one first.
Alibaba Qwen
The default starting point for local work on 8–12 GB cards. Strong instruction following and reliable tool-call formatting for its size.
Alibaba Qwen
The practical local copilot. Supports fill-in-the-middle, so it works as an inline completion model rather than only a chat assistant.
Mistral AI
The model that made local AI practical. Still a dependable, fast, unconditionally Apache-licensed baseline.
Now superseded by Qwen2.5-Coder, but still the reference point for a lot of editor integrations.
Vision plus 128k context on a laptop. The most capable genuinely small multimodal option.