Building a Slack or Teams bot powered by your own LLM starts with the platform's bot framework, the Slack Bolt SDK and Events API for Slack, or the Bot Framework SDK for Microsoft Teams, which handles receiving messages, mentions and thread events from the workspace. That bot server then forwards the conversation to a self-hosted model served through an OpenAI-compatible endpoint, commonly vLLM or NVIDIA NIM, rather than a third-party API, so message content never leaves the company's own infrastructure. For the bot to do more than chat, such as looking up a customer record or creating a ticket, it should call an MCP server exposing those internal actions as tools, keeping the bot logic itself thin. Practical details matter here: handling threading correctly, streaming partial responses so the bot does not appear frozen during longer generations, and rate limiting per channel to avoid runaway costs during busy periods. Nanobase AI builds these bots on top of self-hosted LLM infrastructure it also deploys and operates, so the model, the bot and the internal tool connections form one coherent, privately hosted system.
The message path end to end
A self-hosted LLM chat bot has four distinct hops, and understanding each one matters because a slow or broken link anywhere in the chain shows up to the user as "the bot is stuck."
- The chat platform's event system, Slack's Events API or the Bot Framework SDK for Teams, delivers an incoming message or mention to the bot server.
- The bot server assembles conversation context, typically the current thread's recent messages, and forwards it to the model.
- The model runs on self-hosted infrastructure behind an OpenAI-compatible endpoint, commonly served through vLLM or NVIDIA NIM, rather than a third-party API, keeping message content inside company infrastructure.
- If the model decides an action is needed, a lookup or a ticket creation, the bot server calls an MCP tool layer exposing those internal operations, then returns the combined result to the chat platform.
Keeping the bot server itself thin, doing only message handling and orchestration, while the model and the tool layer stay as separate services, makes each piece independently scalable and testable.
Platform differences that affect the build
| Aspect | Slack | Microsoft Teams |
|---|---|---|
| Event delivery | Events API, webhook-based | Bot Framework SDK, activity-based |
| Threading model | Explicit thread timestamps | Conversation and reply-to-activity IDs |
| Streaming partial responses | Message update via chat.update | Typing indicators plus incremental activity updates |
| Typical auth | OAuth app installed per workspace | Azure Bot registration tied to a Microsoft Entra app |
Neither platform's SDK understands LLM concepts like context windows or tool calls; both only handle message transport, which is why the model and tool layer need to be built and reasoned about separately from the bot framework itself.
Why threading and streaming are not optional
Conversation threads exist for a practical reason beyond UI tidiness: they define what context the bot should send to the model. A bot that ignores threads and includes every recent channel message in every request wastes context window space and risks pulling in unrelated conversations, degrading response quality. Streaming partial output back to the user, updating the same message incrementally rather than waiting for the full generation, matters just as much for perceived responsiveness, since a self-hosted model answering a longer question can take several seconds, and a static "thinking" message reads as broken far sooner than a live-updating one does.
Rate limiting per channel, not just per user
Busy channels can generate bursts of mentions in a short window, especially right after an incident or announcement, and without a channel-level rate limit a single busy channel can consume a disproportionate share of shared GPU capacity, slowing responses for everyone else on the same self-hosted deployment. A practical pattern combines a per-user limit, to stop one person from looping the bot, with a per-channel limit, to cap total load during a spike, and a global queue depth check that degrades gracefully rather than dropping requests silently.
Frequently asked questions
Does the bot need its own MCP server or can it call the internal API directly?
Either works technically. Building it as an MCP server is worth the extra step specifically when other clients, such as an internal dashboard or a second chat platform, will need the same tool access, since it avoids duplicating the integration logic a second time.
How do we prevent the bot from responding to every message in a channel?
Configure the bot to respond only to direct mentions or specific trigger phrases rather than every message, which both platforms support natively through their event filtering, and is the standard pattern for any chat bot regardless of what model sits behind it.
What happens if the self-hosted model is temporarily overloaded?
The bot server should detect a queue timeout or overload response from the inference server and reply with a clear, immediate message rather than leaving the user waiting indefinitely, ideally suggesting a retry or indicating expected wait time if the serving stack exposes queue depth.
How Nanobase AI helps
Nanobase AI builds these bots on top of the same self-hosted LLM infrastructure it deploys and operates, so the model, the bot server and the internal MCP tool connections form one coherently managed system rather than three separately maintained pieces. The serving engine choice behind step three is covered in vllm-vs-tensorrt-llm-vs-ollama-vs-sglang.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.