Agent
Give a local model tools and let it act.
Tool definitions are exposed to the model, calls are parsed from its output and executed under an explicit permission policy. The hard part is not the loop — it is picking a base model whose tool-call formatting is reliable enough to build on.
- The model needs to read files, query a database or call an API
- A task takes several steps and a decision between them
- You want automation that runs against private systems
- Anything where a wrong action is expensive and unattended
Private company agent
An internal assistant that knows your documents, answers in your format, can act on internal systems, and never sends a byte outside the building.
Qwen2.5 7B Instruct
7.6B · 32k · Apache 2.0
The default starting point for local work on 8–12 GB cards. Strong instruction following and reliable tool-call formatting for its size.
Qwen2.5 14B Instruct
14.8B · 32k · Apache 2.0
The sweet spot for 24 GB cards. Meaningfully stronger reasoning than 7B while still fine-tunable locally with QLoRA.
Qwen2.5 32B Instruct
32.8B · 32k · Apache 2.0
Approaches 70B quality at half the memory. Runs on a single 24 GB card at Q4_K_M with a modest context window.
Qwen2.5 72B Instruct
72.7B · 32k · Qwen License
Frontier-adjacent open weights. Needs a workstation, a multi-GPU rig or a large unified-memory Mac.
Qwen3 8B
8.2B · 128k · Apache 2.0
Switchable thinking mode: the same weights answer directly or reason step by step depending on the prompt. Long context for its size.
Qwen3 14B
14.8B · 128k · Apache 2.0
The Qwen3 mid-size. Long context and a reasoning mode inside a footprint a 24 GB card handles comfortably.