Reference

Best models for CPU only — 64 GB workstation

Enough memory to load large models; not enough bandwidth to generate with them at a usable rate.

4 models fit45 GB usableEstimated

Top picks

Best on CPU only — 64 GB workstation

Best overallRuns with CPU offload

Phi-3.5 Mini Instruct

Strong reasoning at 3.8B with a 128k context — but full multi-head attention makes its KV cache expensive.

Params: 3.8B
Quant: Q4_K_M
Memory: 5.7 GB

Runner-upRuns with CPU offload

Llama 3.2 3B Instruct

Small enough for a laptop CPU or a 6 GB card, and still coherent. A sensible target for edge deployment.

Params: 3.2B
Quant: Q4_K_M
Memory: 3.2 GB

ThirdRuns with CPU offload

SmolLM2 1.7B Instruct

Fully open training data and recipe. A good base when you want to understand exactly what the model saw.

Params: 1.7B
Quant: Q4_K_M
Memory: 2.9 GB

Full results

4 models

Estimated

#	Model	Params	Licence	Quantization	Memory	~tok/s	Fit	Fine-tune
1	Phi-3.5 Mini Instruct	3.8B	MIT	Q4_K_M	5.7 GB	40	Runs with CPU offload	No
2	Llama 3.2 3B Instruct	3.2B	Llama 3.2 Community License	Q4_K_M	3.2 GB	48	Runs with CPU offload	No
3	SmolLM2 1.7B Instruct	1.7B	Apache 2.0	Q4_K_M	2.9 GB	90	Runs with CPU offload	No
4	Llama 3.2 1B Instruct	1.24B	Llama 3.2 Community License	Q4_K_M	1.4 GB	124	Runs with CPU offload	No

How this page is produced

Memory is calculated from each model’s published geometry against this machine’s usable capacity. Where the vendor publishes a memory bandwidth figure, an order-of-magnitude token rate is derived from it and labelled as an estimate.