Best local AI models for Radeon RX 7800 XT
16 GB for inference at a good price. Expect to spend time on the ROCm toolchain.
ROCm support on this class is inconsistent; treat local training as experimental.
Specification
VRAM16 GB
Usable for models15 GB
System RAM32 GB
Bandwidth624 GB/s
Local trainingNot practical
Operating systemswindows, linux
Runs well
29 models fit this machine
Estimated
| Model | Params | Quantization | Memory | ~tok/s | Fit | Fine-tune |
|---|---|---|---|---|---|---|
| Phi-4 14B | 14.7B | Q5_K_M | 12 GB | 46 | Good fit | Yes |
| Qwen2.5 14B Instruct | 14.8B | Q5_K_M | 12 GB | 46 | Good fit | Yes |
| Qwen2.5 Coder 7B Instruct | 7.6B | Q8_0 | 8.6 GB | 60 | Excellent fit | Yes |
| StarCoder2 15B | 16B | Q4_K_M | 10 GB | 50 | Good fit | Yes |
| Qwen3 14B | 14.8B | Q4_K_M | 10 GB | 54 | Good fit | Yes |
| Mistral Nemo 12B Instruct | 12.2B | Q5_K_M | 10 GB | 56 | Good fit | Yes |
| DeepSeek-R1-Distill-Qwen-14B | 14.8B | Q5_K_M | 12 GB | 46 | Good fit | Yes |
| Qwen2.5 7B Instruct | 7.6B | Q8_0 | 8.6 GB | 60 | Excellent fit | Yes |
| Llama 3.1 8B Instruct | 8B | Q6_K | 7.7 GB | 74 | Excellent fit | Yes |
| Gemma 2 9B Instruct | 9.2B | Q6_K | 9.9 GB | 64 | Good fit | Yes |
| Gemma 3 12B Instruct | 12.2B | Q5_K_M | 12 GB | 56 | Good fit | Yes |
| DeepSeek-Coder-V2-Lite Instruct | 15.7B | Q4_K_M | 11 GB | 333 | Good fit | Yes |
| Qwen3 8B | 8.2B | Q6_K | 8.0 GB | 72 | Excellent fit | Yes |
| Mistral 7B Instruct v0.3 | 7.25B | Q8_0 | 8.8 GB | 63 | Excellent fit | Yes |
| Mistral Small 24B Instruct | 23.6B | Q3_K_M | 13 GB | 42 | Tight fit | No |
| Code Llama 7B Instruct | 6.7B | Q8_0 | 11 GB | 68 | Good fit | Yes |
| Phi-3.5 Mini Instruct | 3.8B | Q8_0 | 7.3 GB | 119 | Excellent fit | Yes |
| Gemma 3 4B Instruct | 4.3B | Q8_0 | 6.1 GB | 106 | Excellent fit | Yes |
Fine-tunable here