Protecting LLM applications in production typically requires several categories of tools working together rather than one product covering every risk. An LLM gateway or firewall sits between the application and the model to filter prompts and outputs in real time, catching injection attempts, policy violations, and sensitive data before they pass through, with options ranging from commercial platforms to open-source projects such as NeMo Guardrails and Guardrails AI. A dedicated PII detection layer, such as Microsoft's open-source Presidio, screens both inbound prompts and outbound responses for personal data that should never leave or enter the system unmasked. Observability platforms built specifically for LLM applications, such as Langfuse or LangSmith, trace every prompt, retrieval, and response with latency and token usage, which matters for compliance as much as performance because it creates the audit trail regulators and incident responders will ask for. Dedicated red-teaming and adversarial testing tools, including Garak and Microsoft's PyRIT, are used before launch and on an ongoing basis to probe for jailbreaks and injection weaknesses rather than waiting for an incident to reveal them. Choosing among these tools depends on whether the model is self-hosted or API-based, since a self-hosted deployment can run every layer fully offline. Nanobase AI selects and integrates this security tooling based on each client's specific deployment architecture and risk profile.
Map tools to pipeline stages, not marketing categories
Vendor categories like "AI firewall" or "LLM security platform" often bundle capabilities that actually belong to different stages of the request pipeline, which makes it hard to tell what a given tool covers and what it leaves exposed. Mapping each tool category to the specific pipeline stage it protects, ingress, pre-processing, inference, post-processing, or storage, makes gaps in a security stack visible in a way a feature list alone does not.
Tool-to-stage reference
| Pipeline stage | What can go wrong | Tool category | Example tools |
|---|---|---|---|
| Ingress | Malicious or injected prompts entering the system | Prompt filtering gateway | NeMo Guardrails, Guardrails AI |
| Pre-processing | Sensitive data (PII) entering the model unmasked | PII detection and masking | Microsoft Presidio |
| Inference | Jailbreaks, harmful generation, hallucination | Runtime guardrails, output classifiers | NeMo Guardrails, custom classifiers |
| Post-processing | Sensitive data leaking out in the response | Output filtering, PII redaction | Microsoft Presidio, custom output filters |
| Storage and observability | No audit trail for incident response or compliance | Tracing and observability platforms | Langfuse, LangSmith |
| Pre-launch and ongoing testing | Undiscovered jailbreak and injection weaknesses | Adversarial testing and red teaming | Garak, Microsoft PyRIT |
A stack that only implements ingress filtering and a red-teaming pass before launch, without post-processing output filtering or ongoing observability, has a real gap: a jailbreak discovered after the pre-launch test would have no runtime detection or audit trail once the system is live.
Build versus buy at each layer
Ingress and output filtering are increasingly available as managed platforms, which is often the faster path for teams without dedicated security engineering capacity, while observability and adversarial testing frequently work better assembled from open-source components such as Langfuse for tracing and Garak or PyRIT for testing, since these integrate directly with a self-hosted stack without adding another vendor dependency. Self-hosted deployments in particular can run every layer fully offline, which matters for organizations that need to avoid sending prompts to yet another third-party service just to secure the first one.
Evaluating a vendor claim
A five-minute round of these questions filters out most vendors whose actual coverage does not match their marketing category.
- Ask which specific pipeline stage or stages the product actually covers, using the table above as a reference, rather than accepting a general claim of "LLM security."
- Request a sample report or output from the tool against a known test case, not just a feature list.
- Confirm whether the tool works with the specific inference stack in use, such as vLLM or TensorRT-LLM, since some security tools are built around a specific API format.
- Check whether the tool adds its own network dependency, such as calling an external API to classify prompts, which may be unacceptable for a fully self-hosted deployment.
- Verify the tool produces logs or reports suitable for the audit trail a compliance review will require, not just real-time blocking.
Frequently asked questions
Does one platform typically cover every pipeline stage?
Some commercial platforms bundle several stages, particularly ingress and post-processing filtering, but observability and dedicated adversarial testing are frequently separate tools even within an otherwise consolidated security stack, so verifying actual coverage matters more than the platform's marketing scope.
Is open-source tooling sufficient for a regulated industry deployment?
Open-source tools such as NeMo Guardrails, Presidio, and Garak are used in regulated deployments, but the surrounding integration, logging, and retention configuration still needs to be built to the standard the specific regulation requires, since the open-source tool itself does not guarantee compliance.
How does this connect to choosing a red-teaming vendor?
The adversarial testing row in the table above is the pre-launch and ongoing verification layer; choosing a vendor for that specific work is covered in companies offering LLM red teaming.
How Nanobase AI helps
Nanobase AI, a Silicon Valley enterprise AI engineering company, selects and integrates security tooling stage by stage based on each client's specific deployment architecture, whether self-hosted or API-based, closing the gaps a piecemeal tool selection tends to leave open. See a working example in a live demo of a hardened deployment.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.