Reference

Best models for A100 80 GB

Datacentre class. Full fine-tunes of mid-size models and 70B inference at high precision.

29 models fit79 GB usableEstimated

Top picks

Best on A100 80 GB

Best overallGood fit

Qwen2.5 72B Instruct

Frontier-adjacent open weights. Needs a workstation, a multi-GPU rig or a large unified-memory Mac.

Params: 72.7B
Quant: Q5_K_M
Memory: 52 GB

Runner-upGood fit

Llama 3.3 70B Instruct

Delivers close to Llama 3.1 405B quality at a size a dual-GPU workstation or 64 GB Mac can actually hold.

Params: 70.6B
Quant: Q5_K_M
Memory: 51 GB

ThirdExcellent fit

Qwen2.5 Coder 32B Instruct

The strongest open code model that still fits one 24 GB card. The reason a lot of people buy a 4090.

Params: 32.8B
Quant: Q8_0
Memory: 36 GB

Full results

29 models

Estimated

#	Model	Params	Licence	Quantization	Memory	~tok/s	Fit	Fine-tune
1	Qwen2.5 72B Instruct	72.7B	Qwen License	Q5_K_M	52 GB	31	Good fit	Yes
2	Llama 3.3 70B Instruct	70.6B	Llama 3.3 Community License	Q5_K_M	51 GB	32	Good fit	Yes
3	Qwen2.5 Coder 32B Instruct	32.8B	Apache 2.0	Q8_0	36 GB	45	Excellent fit	Yes
4	Qwen2.5 32B Instruct	32.8B	Apache 2.0	Q8_0	36 GB	45	Excellent fit	Yes
5	Mixtral 8x7B Instruct	46.7B	Apache 2.0	Q8_0	49 GB	115	Good fit	Yes
6	DeepSeek-R1-Distill-Qwen-32B	32.8B	MIT	Q8_0	36 GB	45	Excellent fit	Yes
7	Gemma 3 27B Instruct	27.4B	Gemma Terms of Use	Q8_0	33 GB	54	Excellent fit	Yes
8	Phi-4 14B	14.7B	MIT	F16	30 GB	54	Excellent fit	Yes
9	Mistral Small 24B Instruct	23.6B	Apache 2.0	F16	46 GB	33	Excellent fit	Yes
10	Qwen2.5 14B Instruct	14.8B	Apache 2.0	F16	30 GB	53	Excellent fit	Yes
11	Qwen3 30B-A3B	30.5B	Apache 2.0	F16	58 GB	239	Good fit	Yes
12	StarCoder2 15B	16B	BigCode OpenRAIL-M	F16	31 GB	49	Excellent fit	Yes
13	Qwen3 14B	14.8B	Apache 2.0	F16	30 GB	53	Excellent fit	Yes
14	DeepSeek-R1-Distill-Qwen-14B	14.8B	MIT	F16	30 GB	53	Excellent fit	Yes
15	Mistral Nemo 12B Instruct	12.2B	Apache 2.0	F16	25 GB	65	Excellent fit	Yes
16	Qwen2.5 Coder 7B Instruct	7.6B	Apache 2.0	F16	15 GB	104	Excellent fit	Yes
17	Gemma 3 12B Instruct	12.2B	Gemma Terms of Use	F16	26 GB	65	Excellent fit	Yes
18	DeepSeek-Coder-V2-Lite Instruct	15.7B	DeepSeek License	F16	32 GB	328	Excellent fit	Yes
19	Gemma 2 9B Instruct	9.2B	Gemma Terms of Use	F16	20 GB	86	Excellent fit	Yes
20	Qwen2.5 7B Instruct	7.6B	Apache 2.0	F16	15 GB	104	Excellent fit	Yes
21	Llama 3.1 8B Instruct	8B	Llama 3.1 Community License	F16	16 GB	99	Excellent fit	Yes
22	Qwen3 8B	8.2B	Apache 2.0	F16	17 GB	96	Excellent fit	Yes
23	Mistral 7B Instruct v0.3	7.25B	Apache 2.0	F16	15 GB	109	Excellent fit	Yes
24	Code Llama 7B Instruct	6.7B	Llama 2 Community License	F16	17 GB	118	Excellent fit	Yes
25	Phi-3.5 Mini Instruct	3.8B	MIT	F16	11 GB	207	Excellent fit	Yes
26	Gemma 3 4B Instruct	4.3B	Gemma Terms of Use	F16	9.9 GB	183	Excellent fit	Yes
27	Llama 3.2 3B Instruct	3.2B	Llama 3.2 Community License	F16	7.3 GB	246	Excellent fit	Yes
28	SmolLM2 1.7B Instruct	1.7B	Apache 2.0	F16	5.2 GB	464	Excellent fit	Yes
29	Llama 3.2 1B Instruct	1.24B	Llama 3.2 Community License	F16	3.0 GB	636	Excellent fit	Yes

How this page is produced

Memory is calculated from each model’s published geometry against this machine’s usable capacity. Where the vendor publishes a memory bandwidth figure, an order-of-magnitude token rate is derived from it and labelled as an estimate.

· 29 of these can also be fine-tuned on this hardware.

Check your own machine

ModelLM can read your actual GPU, VRAM and RAM and size every model against it.

Detect my hardware

Best models for GeForce RTX 5090 Best models for GeForce RTX 4090 Best models for GeForce RTX 5080 Best models for GeForce RTX 4080 Super Best models for GeForce RTX 3090