Best local AI models for CPU only — 64 GB workstation

Enough memory to load large models; not enough bandwidth to generate with them at a usable rate.

Filter the catalogue Use my real machine

Specification

VRAM64 GB

Usable for models45 GB

System RAM64 GB

Bandwidth120 GB/s

Local trainingNot practical

Operating systemswindows, linux

Runs well

4 models fit this machine

Estimated

Model	Params	Quantization	Memory	~tok/s	Fit	Fine-tune
Phi-3.5 Mini Instruct	3.8B	Q4_K_M	5.7 GB	40	Runs with CPU offload	No
Llama 3.2 3B Instruct	3.2B	Q4_K_M	3.2 GB	48	Runs with CPU offload	No
SmolLM2 1.7B Instruct	1.7B	Q4_K_M	2.9 GB	90	Runs with CPU offload	No
Llama 3.2 1B Instruct	1.24B	Q4_K_M	1.4 GB	124	Runs with CPU offload	No

Fine-tuning

Nothing in the catalogue can be fine-tuned on this machine. Local training needs an accelerator with enough memory to hold the base weights, the adapter, the optimizer state and the activations at once.

Best local AI models for CPU only — 64 GB workstation

Enough memory to load large models; not enough bandwidth to generate with them at a usable rate.

Filter the catalogue Use my real machine

Specification

VRAM64 GB

Usable for models45 GB

System RAM64 GB

Bandwidth120 GB/s

Local trainingNot practical

Operating systemswindows, linux

Runs well

4 models fit this machine

Estimated

Model	Params	Quantization	Memory	~tok/s	Fit	Fine-tune
Phi-3.5 Mini Instruct	3.8B	Q4_K_M	5.7 GB	40	Runs with CPU offload	No
Llama 3.2 3B Instruct	3.2B	Q4_K_M	3.2 GB	48	Runs with CPU offload	No
SmolLM2 1.7B Instruct	1.7B	Q4_K_M	2.9 GB	90	Runs with CPU offload	No
Llama 3.2 1B Instruct	1.24B	Q4_K_M	1.4 GB	124	Runs with CPU offload	No

Fine-tuning

Best local AI models for CPU only — 64 GB workstation

4 models fit this machine

25 models will not run here

Best local AI models for CPU only — 64 GB workstation

4 models fit this machine

25 models will not run here