Building an accurate AI support chatbot requires grounding every response in a retrieval system over your actual help center, policies and order data, rather than relying on the model's general training knowledge. The typical architecture pairs a retrieval-augmented generation pipeline, which indexes your documentation and past resolved tickets in a vector database, with a large language model instructed to answer only from the retrieved passages and to say it does not know rather than guess. Accuracy also depends on evaluation discipline: build a test set of at least a few hundred real customer questions with verified correct answers, measure the bot's responses against them before launch, and re-run that suite whenever the model, prompts or knowledge base change. Citing source articles in every answer, setting confidence thresholds that trigger a human handoff, and scoping the system prompt tightly to your domain further reduce made-up answers. Most teams also add a guardrail layer that blocks the bot from discussing pricing changes, legal commitments or anything outside its verified knowledge. Nanobase AI, a Silicon Valley enterprise AI engineering company, builds this retrieval, evaluation and guardrail stack end to end so support answers stay accurate as content changes.
Accuracy is a process, not a launch checklist
Most teams treat chatbot accuracy as something you achieve once, at launch, by wiring up retrieval-augmented generation and writing a careful system prompt. That gets a bot into production, but it does not keep it accurate six months later when the help center gets rewritten, a new product line launches, or the underlying model gets swapped for a cheaper one. The teams whose chatbots stay accurate are the ones that treat evaluation as a running pipeline that gates every change, not a one-time pre-launch test.
What a golden test set actually needs
A golden test set is a curated list of real customer questions paired with a verified correct answer and, ideally, the source document that answer should come from. Building one well means:
- Pull 200 to 500 real questions from historical tickets, weighted toward your highest-volume categories rather than edge cases.
- Have a subject-matter expert, not the engineering team, write or approve the correct answer for each one.
- Tag each question with the policy area it touches (refunds, billing, technical troubleshooting) so failures can be grouped by category.
- Include a deliberate slice of unanswerable questions, ones your documentation genuinely does not cover, so you can measure whether the bot correctly says it does not know.
- Re-run the full set before every prompt change, model swap, or knowledge base migration, and store results so regressions are visible over time.
A test set without a subject-matter expert's verified answer and a tagged policy category is just a list of questions, not a usable regression gate.
Grading responses without drowning in manual review
Manual grading of every test run does not scale past a few dozen questions. Most production pipelines use a second LLM as a grader, comparing the bot's answer against the golden answer for factual consistency, with human review reserved for a random sample and for anything the grader flags as uncertain. This LLM-as-judge approach is imperfect but consistent, which matters more than perfection when the goal is catching regressions between versions rather than producing a definitive accuracy score.
Where accuracy actually breaks in production
| Failure mode | Typical cause | Mitigation |
|---|---|---|
| Confidently wrong answer | Retrieval returned a near-miss document, model filled gaps | Stricter relevance threshold, explicit "say you don't know" instruction |
| Outdated answer | Knowledge base updated, index not refreshed | Automated re-indexing on document change, not a manual batch job |
| Right fact, wrong policy version | Multiple similar documents (old and new pricing) in the index | Deduplication and expiry dates on retrieved documents |
| Answer contradicts a human agent's prior reply | No shared source of truth between bot and agents | Route both through the same knowledge base and ticketing history |
Most hallucinations trace back to a retrieval or content problem, not a model problem, which is why fixing the knowledge base usually does more than switching models.
Building the human review loop
Automated grading catches known failure patterns; it does not catch the ones you have not thought of yet. A sustainable review loop samples a fixed percentage of live conversations weekly, has a support lead flag anything wrong or borderline, and feeds confirmed errors back into the golden test set so the same mistake cannot silently recur. The review loop's real value is catching a subtler problem automated grading misses: a bot that is technically accurate per your test set but that customers still find unhelpful because it answers narrowly or misses the real intent behind a question.
Frequently asked questions
How many test questions do we need before launch?
A useful starting point is 200 to 300 questions covering your top ticket categories, though the exact number matters less than making sure high-volume categories are well represented and that a subject-matter expert, not an engineer, verifies each correct answer.
Should we use the same LLM to grade its own answers?
Avoid using the identical model and prompt to both answer and grade, since it tends to rate its own mistakes favorably; a different model, or the same model with a distinct grading prompt and no memory of having answered, gives a more honest signal.
How often should the eval suite run?
Run it on every prompt or model change as a release gate, and on a schedule, weekly is common, to catch knowledge base drift even when nothing in the code changed.
What is a reasonable accuracy target before launch?
There is no universal number, since it depends on ticket complexity and risk tolerance, but most teams require near-total accuracy on their top question categories before allowing the bot to answer them without a confidence check.
How Nanobase AI helps
Nanobase AI builds the retrieval, grounding and evaluation pipeline together rather than treating testing as an afterthought, including the golden test set, the LLM-as-judge grading harness, and the live sampling loop that catches drift after launch. This work is often scoped alongside a broader AI agents and automation engagement, and pairs well with fixing chatbot hallucination at the architecture level rather than patching individual wrong answers. You can see the retrieval and evaluation approach in a live demo before committing to a build.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.