Best overallGood fit
Qwen2.5 Coder 7B Instruct
The practical local copilot. Supports fill-in-the-middle, so it works as an inline completion model rather than only a chat assistant.
- Params
- 7.6B
- Quant
- Q6_K
- Memory
- 6.8 GB
Reference
Best models for GeForce RTX 4070
A solid 7–9B machine. 14B fits at Q4_K_M with a short context and no room for much else.
27 models fit11 GB usableEstimated
Top picks
Best on GeForce RTX 4070
Runner-upGood fit
Gemma 2 9B Instruct
Unusually good at natural, well-structured prose for its size. The short context is the catch.
- Params
- 9.2B
- Quant
- Q4_K_M
- Memory
- 8.1 GB
ThirdGood fit
Qwen2.5 7B Instruct
The default starting point for local work on 8–12 GB cards. Strong instruction following and reliable tool-call formatting for its size.
- Params
- 7.6B
- Quant
- Q6_K
- Memory
- 6.8 GB
Full results
27 models
Estimated
| # | Model | Params | Licence | Quantization | Memory | ~tok/s | Fit | Fine-tune |
|---|---|---|---|---|---|---|---|---|
| 1 | Qwen2.5 Coder 7B Instruct | 7.6B | Apache 2.0 | Q6_K | 6.8 GB | 63 | Good fit | Yes |
| 2 | Gemma 2 9B Instruct | 9.2B | Gemma Terms of Use | Q4_K_M | 8.1 GB | 70 | Good fit | Yes |
| 3 | Qwen2.5 7B Instruct | 7.6B | Apache 2.0 | Q6_K | 6.8 GB | 63 | Good fit | Yes |
| 4 | Llama 3.1 8B Instruct | 8B | Llama 3.1 Community License | Q6_K | 7.7 GB | 59 | Good fit | Yes |
| 5 | Mistral 7B Instruct v0.3 | 7.25B | Apache 2.0 | Q4_K_M | 5.7 GB | 89 | Excellent fit | Yes |
| 6 | Qwen3 8B | 8.2B | Apache 2.0 | Q5_K_M | 7.1 GB | 67 | Good fit | Yes |
| 7 | Code Llama 7B Instruct | 6.7B | Llama 2 Community License | Q4_K_M | 8.3 GB | 96 | Good fit | Yes |
| 8 | StarCoder2 15B | 16B | BigCode OpenRAIL-M | Q4_K_M | 10 GB | 40 | Tight fit | No |
| 9 | Mistral Nemo 12B Instruct | 12.2B | Apache 2.0 | Q4_K_M | 9.1 GB | 53 | Tight fit | Yes |
| 10 | Qwen3 14B | 14.8B | Apache 2.0 | Q4_K_M | 10 GB | 44 | Tight fit | No |
| 11 | Phi-3.5 Mini Instruct | 3.8B | MIT | Q4_K_M | 5.7 GB | 170 | Excellent fit | Yes |
| 12 | Gemma 3 4B Instruct | 4.3B | Gemma Terms of Use | Q5_K_M | 4.7 GB | 128 | Excellent fit | Yes |
| 13 | Gemma 3 12B Instruct | 12.2B | Gemma Terms of Use | Q4_K_M | 10 GB | 53 | Tight fit | Yes |
| 14 | Llama 3.2 3B Instruct | 3.2B | Llama 3.2 Community License | Q8_0 | 4.6 GB | 115 | Excellent fit | Yes |
| 15 | Qwen2.5 Coder 32B Instruct | 32.8B | Apache 2.0 | Q4_K_M | 21 GB | 20 | Runs with CPU offload | No |
| 16 | Qwen2.5 32B Instruct | 32.8B | Apache 2.0 | Q4_K_M | 21 GB | 20 | Runs with CPU offload | No |
| 17 | Mixtral 8x7B Instruct | 46.7B | Apache 2.0 | Q4_K_M | 29 GB | 50 | Runs with CPU offload | No |
| 18 | SmolLM2 1.7B Instruct | 1.7B | Apache 2.0 | F16 | 5.2 GB | 115 | Excellent fit | Yes |
| 19 | DeepSeek-R1-Distill-Qwen-32B | 32.8B | MIT | Q4_K_M | 21 GB | 20 | Runs with CPU offload | No |
| 20 | Llama 3.2 1B Instruct | 1.24B | Llama 3.2 Community License | F16 | 3.0 GB | 157 | Excellent fit | Yes |
| 21 | Gemma 3 27B Instruct | 27.4B | Gemma Terms of Use | Q4_K_M | 21 GB | 24 | Runs with CPU offload | No |
| 22 | Phi-4 14B | 14.7B | MIT | Q4_K_M | 11 GB | 44 | Runs with CPU offload | No |
| 23 | Mistral Small 24B Instruct | 23.6B | Apache 2.0 | Q4_K_M | 16 GB | 27 | Runs with CPU offload | No |
| 24 | Qwen2.5 14B Instruct | 14.8B | Apache 2.0 | Q4_K_M | 11 GB | 44 | Runs with CPU offload | No |
| 25 | Qwen3 30B-A3B | 30.5B | Apache 2.0 | Q4_K_M | 19 GB | 196 | Runs with CPU offload | No |
| 26 | DeepSeek-R1-Distill-Qwen-14B | 14.8B | MIT | Q4_K_M | 11 GB | 44 | Runs with CPU offload | No |
| 27 | DeepSeek-Coder-V2-Lite Instruct | 15.7B | DeepSeek License | Q4_K_M | 11 GB | 269 | Runs with CPU offload | Yes |
How this page is produced
Memory is calculated from each model’s published geometry against this machine’s usable capacity. Where the vendor publishes a memory bandwidth figure, an order-of-magnitude token rate is derived from it and labelled as an estimate.
- · 15 of these can also be fine-tuned on this hardware.
Check your own machine
ModelLM can read your actual GPU, VRAM and RAM and size every model against it.
Detect my hardwareRelated