Instruction tuning and domain adaptation solve different problems and are often used together rather than as alternatives. Instruction tuning, typically done through supervised fine-tuning on instruction-response pairs, teaches a model how to follow directions, format its output correctly and behave in a task-appropriate way, without necessarily changing what the model knows about any particular subject. Domain adaptation, most reliably achieved through continued pretraining on a large volume of raw domain text, changes what the model knows by shifting its internal representations toward a specific field's vocabulary, writing style and factual content, such as legal, medical or financial language. A model can be excellent at following instructions in general while still being weak on a specialized domain's terminology, and conversely a domain-adapted model that has only seen raw text may not yet know how to structure a helpful response, which is why the common enterprise pattern is domain adaptation first to build knowledge, followed by instruction tuning to teach the model how to apply that knowledge in a useful conversational or task format. Nanobase AI sequences these two techniques deliberately rather than treating fine-tuning as a single undifferentiated step.
Two different questions, easy to conflate
"Teach the model to behave correctly" and "teach the model to know our field" sound similar but require different techniques, different data, and different amounts of it, and conflating them is a common source of disappointing fine-tuning results, where a team applies instruction tuning to a problem that actually needed domain knowledge the base model never had. Instruction tuning, done through supervised fine-tuning on instruction-response pairs, teaches a model how to follow directions, format output correctly, and behave in a task-appropriate way, largely independent of what factual or terminological knowledge it draws on. Domain adaptation, most reliably achieved through continued pretraining on a large volume of raw domain text, changes what the model actually knows by shifting its internal representations toward a field's vocabulary, writing style and factual content. A model can be excellent at following instructions while knowing nothing useful about your specific domain, and equally, a model can have deep domain knowledge while being poor at following the specific instructions your application needs; these are genuinely separate capabilities that need separate solutions.
Comparing the two directly
Data volume is the starkest contrast in this table: domain adaptation needs orders of magnitude more raw text than instruction tuning needs structured examples.
| Dimension | Instruction tuning | Domain adaptation |
|---|---|---|
| What it changes | How the model behaves and formats output | What the model knows |
| Typical technique | Supervised fine-tuning on instruction-response pairs | Continued pretraining on raw domain text |
| Data needed | Structured examples, hundreds to low thousands | Large volume of raw text, hundreds of millions of tokens or more |
| Data format | Instruction/response pairs | Raw, unstructured domain documents |
| Typical use case | Task-specific behavior, format compliance, tone | Legal, medical, financial or scientific terminology and reasoning patterns |
| Compute intensity | Lower, especially with LoRA | Higher; continued pretraining processes far more tokens |
The recipe most enterprise domain-specific models actually need
Skipping straight to instruction tuning when a genuine knowledge gap exists is the single most common reason a domain-specific fine-tuning project underdelivers.
- Assess whether the base model already has reasonable domain knowledge from its original pretraining, since many current frontier open-weight models have broad exposure to common professional domains and may need only instruction tuning rather than full domain adaptation.
- If genuine knowledge gaps exist, specific terminology, niche factual content, or domain-specific reasoning patterns absent from general pretraining, run continued pretraining on a substantial corpus of raw domain text before any instruction tuning.
- Follow continued pretraining with instruction tuning on task-specific examples, since domain adaptation alone does not teach the model how to apply that knowledge in the format and style your application needs.
- Evaluate the two stages separately where possible: test domain knowledge with factual and terminology-based questions, and test instruction-following with format and task-completion checks, since a single blended evaluation can mask which stage needs more work.
- Consider whether RAG could substitute for domain adaptation for knowledge that changes frequently, since continued pretraining bakes knowledge into weights that becomes stale as the domain's facts change, while RAG retrieves current information at query time.
Domain examples where this distinction matters concretely
A legal document review task might need domain adaptation to internalize legal terminology and citation conventions, combined with instruction tuning to produce output in the specific structured format the review workflow requires. A medical coding assistant likely needs both stronger domain-specific vocabulary understanding, gained through continued pretraining or a domain-adapted base model, and instruction tuning to reliably output codes in the exact required format. A customer support model, by contrast, usually needs comparatively little domain adaptation since the language involved is closer to general English, and gets most of its value from instruction tuning on your specific policies and phrasing, which connects to the considerations in fine-tuning for brand voice and tone.
Frequently asked questions
Do we always need continued pretraining for a domain-specific model?
No, only when the base model genuinely lacks sufficient domain knowledge from its original pretraining. Many current open-weight models already have broad exposure to common professional domains, and instruction tuning alone is often sufficient; continued pretraining is worth the added compute investment specifically for narrow or unusual domains poorly represented in general pretraining data.
Can instruction tuning teach a model new facts?
Generally not reliably at scale; instruction tuning is much better suited to teaching behavior, format and style than to reliably encoding large amounts of new factual knowledge, since instruction datasets are typically far smaller than what would be needed to shift the model's factual knowledge base meaningfully.
Is domain adaptation the same as fine-tuning on domain-specific instruction data?
No, domain adaptation specifically refers to continued pretraining on raw, unstructured domain text to shift the model's general knowledge, while fine-tuning on domain-specific instruction data is a form of instruction tuning that happens to use domain-relevant examples; the two are complementary but technically distinct processes.
How Nanobase AI helps
Nanobase AI designs combined domain adaptation and instruction tuning pipelines for enterprises in specialized fields like insurance, finance and legal, sequencing continued pretraining and instruction tuning correctly rather than defaulting to instruction tuning alone when genuine domain adaptation is needed. See our insurance AI work for an example of this combined approach in practice.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.