Automatic document classification works by training or prompting a model to assign each incoming document to a predefined category, such as invoice, contract, resume or complaint, based on its text content, layout and visual features, then routing it to the right workflow or team without manual sorting. Two main approaches exist: a fine-tuned classifier, often built on a smaller language model, trained on labeled examples of each document type, which performs well when categories are stable and volume justifies the training effort, and zero-shot or few-shot classification using a large language model prompted with the category definitions, which works immediately without labeled data but costs more per document at scale. Layout and visual cues, such as a logo, letterhead or standard form structure, often improve accuracy beyond text alone, especially for scanned documents where OCR errors can confuse a text-only classifier. A confidence threshold should route uncertain classifications to a human reviewer rather than force every document into a category automatically. Classification accuracy should be tracked per category, since a rare document type is far more likely to be misclassified than a common one. Nanobase AI, a Silicon Valley enterprise AI engineering company, builds classification pipelines sized to a customer's document volume and category count rather than a generic one-model solution.
Taxonomy design comes before model selection
The most common mistake in a document classification project is choosing a model before defining the category taxonomy carefully, when the taxonomy itself is usually what determines accuracy. Categories that overlap in practice, such as "invoice" and "credit note," or a catch-all "other" bucket that ends up absorbing a large share of volume, cause more misclassification than any model limitation, because no classifier can reliably distinguish categories a business itself has not clearly separated. A well-designed taxonomy defines each category with concrete, distinguishing features, not just a name, and treats rare document types as an explicit "needs review" category rather than forcing them into the nearest available label.
Revisiting the taxonomy after the first few weeks of production volume, once real document variety is visible, usually improves accuracy more than any amount of model tuning against the original category list.
Comparing classifier approaches
| Approach | Setup effort | Cost at scale | Best fit |
|---|---|---|---|
| Keyword or rule-based | Low | Very low | Small number of clearly distinct categories |
| Fine-tuned small classifier | Medium (needs labeled data) | Low per document | Stable categories, high volume |
| Zero-shot LLM prompting | Very low | Higher per document | New or evolving categories, lower volume |
| Few-shot LLM prompting | Low | Medium per document | Balance of setup speed and accuracy |
Fine-tuned classifiers cost less per document once trained, which matters at high volume, but they need labeled examples and retraining when categories change. Zero-shot and few-shot approaches using a large language model start working immediately with no labeled data, which suits a new document type or a taxonomy still being refined, at a higher per-document cost.
Using layout and visual cues, not just text content
Text-only classification struggles when OCR introduces errors on a scanned document, or when two document types share very similar wording but differ in structure, such as a purchase order and an invoice from the same vendor. Incorporating layout and visual features, a logo, letterhead position, a standard form structure, or the presence of a signature block, often improves accuracy beyond what text content alone provides, particularly for scanned documents where the text itself is noisy. Vision-language models that classify directly from a page image, considering both text and layout together, tend to outperform a text-only classifier on exactly this category of ambiguous, visually distinct document types.
Monitoring for drift after launch
Classification accuracy that looked strong during testing can degrade over months as new vendors, form versions or document sources appear that were not represented in the original training or prompt examples. Tracking accuracy per category on an ongoing basis, not just at launch, surfaces which specific category is drifting rather than waiting for a generic accuracy complaint to surface the problem. A confidence threshold that routes uncertain classifications to a human reviewer, rather than forcing every document into a category, both limits the damage from drift and generates a steady stream of labeled correction data that can retrain or refine the system.
Frequently asked questions
How many categories should a classification taxonomy have?
As few as the business genuinely needs to route documents differently, since every additional category increases the chance of overlap and misclassification; most production systems work best with a focused set of well-separated categories plus an explicit review category for anything unclear.
Does classification need to happen before or after OCR?
Layout and visual classification can happen on the raw image before OCR, which is often faster and avoids compounding OCR errors into the classification step; text-based classification, by contrast, requires OCR or native text extraction to run first before it can begin.
What is a reasonable confidence threshold for routing to human review?
There is no universal number; it depends on the cost of a misclassification in a specific workflow. A threshold should be set by testing on real documents and observing where the tradeoff between review volume and misclassification rate matches the business's risk tolerance.
How Nanobase AI helps
Nanobase AI, a Silicon Valley enterprise AI engineering company, designs classification taxonomies and selects between fine-tuned and LLM-based classifiers based on a customer's actual document volume and category stability, not a default approach. See Nanobase AI's solutions for how this fits a broader automation build. Related: sorting and routing email attachments with AI.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.