Qwen 3 and Llama 4 currently rank among the strongest open-weight models for function calling and tool use, both scoring competitively with proprietary models on the Berkeley Function-Calling Leaderboard, which tests whether a model produces correctly structured, executable calls across single, parallel and multi-turn scenarios. Qwen 3 was trained with explicit agentic and tool-use data and handles nested or parallel tool calls reliably, which matters for agents that must call several APIs in one turn. Llama 4's larger context window helps when tool definitions and prior call history need to stay in the prompt across a long agent session. DeepSeek V3 also performs well but has historically needed more careful prompt engineering to keep output strictly in the expected JSON schema. In practice, raw benchmark rank matters less than testing against your exact tool schema, since real failures usually come from edge cases like optional parameters, ambiguous tool selection among similar functions, or malformed output under load. Nanobase AI, a Silicon Valley enterprise AI engineering company, builds and hardens the agent orchestration layer around whichever model proves most reliable on a client's actual tool set.
Leaderboard rank is a starting filter, not a guarantee
The Berkeley Function-Calling Leaderboard is a useful way to shortlist models, since it tests structured, executable tool calls across single, parallel and multi-turn scenarios in a standardized way. But leaderboard scores are measured against a fixed set of test tools and schemas, and real production agents fail in more specific ways: an ambiguous tool choice among several similarly named functions, an optional parameter the model omits incorrectly, or malformed JSON that only appears under concurrent load. None of these show up reliably in a single aggregate score.
Use the leaderboard to build a shortlist of two or three candidates, then test the specific failure modes your own tool schema is likely to trigger.
Failure modes worth testing deliberately
| Failure mode | How to test it | Why it matters in production |
|---|---|---|
| Ambiguous tool selection | Provide two tools with overlapping purposes | Real tool sets often have near-duplicate functions |
| Missing optional parameters | Omit optional fields from the tool schema | Models sometimes hallucinate values instead of omitting them |
| Parallel call correctness | Ask for a task requiring two independent tool calls at once | Not all models handle simultaneous calls reliably |
| Malformed output under load | Run concurrent requests and check JSON validity | Schema adherence can degrade under batching pressure |
| Multi-turn state tracking | Chain several tool calls across a conversation | Context about prior calls can be lost over long sessions |
A model that scores well on isolated single-call tests can still fail in production once ambiguity, concurrency and multi-turn state are introduced together.
Constrained decoding closes the remaining gap
Even a strong model occasionally produces output that does not match the exact tool schema, and for production agents this residual error rate is often unacceptable on its own. Pairing the model with constrained decoding, using tools like vLLM's structured output support or a JSON schema grammar, guarantees syntactically valid tool calls regardless of the model's own reliability, shifting remaining errors from malformed output to semantic mistakes that are easier to catch and handle downstream.
Constrained decoding should be standard for any production tool-calling agent, since it eliminates a category of failure no model choice alone fully solves.
Design the fallback path before launch
No model reaches perfect tool-calling accuracy, so a production agent needs an explicit fallback when a tool call fails validation or returns an unexpected result: retrying with clarified instructions, falling back to a direct text response, or escalating to a human. Deciding this behavior during evaluation, rather than discovering it in production, avoids agents that silently fail or loop on a broken tool call.
Plan the fallback behavior for tool-calling failures as part of the evaluation phase, not as an afterthought once the agent is already live.
Frequently asked questions
Does Qwen 3 or Llama 4 handle parallel tool calls better?
Both perform competitively, with Qwen 3's agentic training data giving it an edge on nested and parallel calls in independent testing, while Llama 4's larger context window helps when many tool definitions and prior call history must stay in the prompt across a long session. Test both against your specific tool set before deciding.
Is DeepSeek V3 a good choice for tool-calling agents?
It can be, though it has historically needed more careful prompt engineering to keep output strictly within a JSON schema compared to Qwen 3 and Llama 4. Pairing it with constrained decoding mitigates most of this gap.
How many test cases are enough to trust a function-calling evaluation?
Aim for at least 30 to 50 cases covering the failure modes above, weighted toward the tool combinations your agent will actually use most. A handful of happy-path tests is not enough to catch the edge cases that cause production incidents.
How Nanobase AI helps
Nanobase AI, a Silicon Valley enterprise AI engineering company, builds and hardens the agent orchestration layer, including constrained decoding and fallback handling, around whichever model proves most reliable on a client's actual tool set. Learn about our approach to MCP servers and tool integration or explore our solutions.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.