They solve different problems
Retrieval changes what is in the prompt. Fine-tuning changes what is in the weights. That single distinction settles most cases: a model that does not know your product catalogue needs the catalogue in its context, not a week of gradient descent.
Fine-tuning is for behaviour that no amount of prompting will hold — a rigid output structure, a house voice across a whole team, a task the model has genuinely never seen performed. It is a poor way to install facts, because models memorise inconsistently and still hallucinate around the edges of what they memorised.
| RAG | Fine-tuning | |
|---|---|---|
| Changes | The prompt | The weights |
| Good for | Facts, documents, citations | Format, tone, task behaviour |
| Update cost | Reindex — minutes | Retrain and re-evaluate — hours |
| Needs a GPU | No | Yes |
| Traceable answers | Yes, cite the chunk | No |
| Minimum data | One document | ~200 examples, ideally 1,000+ |
| Fails by | Retrieving the wrong passage | Overfitting, capability regression |
How much data before fine-tuning is worth it
Below roughly 200 clean instruction/response pairs, a carefully written system prompt almost always wins. Between 200 and 1,000 a LoRA starts to beat prompting on narrow, consistent tasks. Above 3,000 it wins convincingly, assuming the examples are actually good.
Quality matters more than volume. Five hundred consistent examples beat five thousand that disagree with each other about format — training on contradictory examples teaches the contradiction.
Documents are not a training dataset
A folder of PDFs is a knowledge base. Fine-tuning needs input/output pairs — what someone asked, and what a good answer looked like. Converting documents into training pairs is its own piece of work, and usually not worth doing.
When to do both
Both is the right answer when you have a document corpus and enough examples to justify each technique separately. Train the behaviour, retrieve the facts: the adapter learns your response format and domain vocabulary, retrieval supplies the specifics that change.
It is also twice the work, and a regression becomes harder to attribute. Do not reach for it first.
How ModelLM decides
The advisor scores all four approaches against facts measured from your own upload: document count, token volume, example count, dataset quality, duplicate ratio, and how volatile you said the material is. It shows the evidence and the trade-offs, then lets you pick anything — including the option it argued against.
- Documents but no instruction pairs
- Material that changes weekly or faster
- A need to cite sources
- Under a few hundred documents
ModelLM applies these rules to your own data.
Upload what you have and the advisor scores all four approaches against it, with the evidence behind the recommendation.
Create a model