Best local AI models for CPU only — 32 GB DDR5

No GPU. Small models are usable for batch work; anything interactive will feel slow. Retrieval still works well.

CPU fine-tuning is not practical at any useful model size.

Specification

VRAM32 GB

Usable for models22 GB

System RAM32 GB

Bandwidth83 GB/s

Local trainingNot practical

Operating systemswindows, linux, macos

Runs well

4 models fit this machine

Estimated

Model	Params	Quantization	Memory	~tok/s	Fit	Fine-tune
Phi-3.5 Mini Instruct	3.8B	Q4_K_M	5.7 GB	28	Runs with CPU offload	No
Llama 3.2 3B Instruct	3.2B	Q4_K_M	3.2 GB	33	Runs with CPU offload	No
SmolLM2 1.7B Instruct	1.7B	Q4_K_M	2.9 GB	63	Runs with CPU offload	No
Llama 3.2 1B Instruct	1.24B	Q4_K_M	1.4 GB	86	Runs with CPU offload	No

Fine-tuning

Nothing in the catalogue can be fine-tuned on this machine. CPU fine-tuning is not practical at any useful model size.