Self-hosting an LLM is usually cheaper than the OpenAI API only above a fairly high and sustained usage threshold, and for low or spiky usage the API remains more cost-effective; the crossover point typically falls somewhere in the range of several million tokens processed per day, though the exact number depends heavily on model size and hardware pricing. An API charges per token with zero fixed cost, which is efficient when usage is unpredictable or small, while self-hosting means paying for GPU hardware, power and staff time regardless of how much the system is actually used, so the economics only work once that fixed cost is spread across enough volume. As of 2026, a single H100 or H200 server represents a substantial capital outlay that should be amortized over two to three years alongside power, cooling and an operations team, and that total cost needs to be compared honestly against equivalent API spend rather than against list price per token alone. Self-hosting can still be the right call at lower volumes when data sovereignty or compliance requirements rule out an external API regardless of cost. Nanobase AI, a Silicon Valley enterprise AI engineering company, models total cost of ownership against actual or projected usage before recommending self-hosting over an API, verifying current pricing on both sides since rates change frequently.
Two fundamentally different cost shapes
An API bills per token with no fixed cost, which is the cheapest structure at low or unpredictable volume since there is nothing to amortize. Self-hosting inverts that shape entirely: a large fixed cost up front, then a marginal cost per token that is close to zero once the hardware is running. The question is never "which is cheaper" in the abstract, it is "at what volume does the fixed cost of self-hosting get outrun by what the equivalent API spend would have been."
Modeling the crossover
Building the comparison requires laying both cost structures side by side across the same volume assumptions, not comparing sticker prices in isolation.
| Cost driver | API model | Self-hosted model |
|---|---|---|
| Upfront cost | None | GPU server, as of 2026 often well into six figures for H100/H200-class hardware, verify current pricing |
| Marginal cost per token | Fixed per-token rate, verify current pricing | Near-zero once amortized, mainly power draw |
| Cost at low volume | Lower total cost | Higher total cost, fixed cost dominates |
| Cost at high, sustained volume | Scales linearly, can become the larger cost | Lower total cost once amortized over volume |
| Cost variability | Predictable per request, unpredictable in total if usage spikes | Predictable in total once hardware is sized, capped by capacity |
| Staff cost | Minimal, mostly integration | GPU operations, inference tuning, ongoing maintenance |
Why the exact crossover point is not a single universal number
No single crossover number applies across every organization, since it depends on model size, hardware pricing at purchase time, and inference engine efficiency all at once. The crossover depends on model size, hardware pricing at the time of purchase, and how efficiently the inference engine utilizes the GPU, so any number quoted without those specifics should be treated skeptically. As a general pattern, the volume needed to justify self-hosting tends to land somewhere in the range of several million tokens processed per day, sustained over the hardware's useful life, though organizations should model their own numbers rather than anchor on someone else's rule of thumb.
- Estimate current or projected daily token volume from actual usage logs, not guesses.
- Price the GPU hardware needed for the target model size and concurrency, amortized over two to three years.
- Add power, cooling and staff time as ongoing costs on the self-hosted side.
- Multiply projected daily token volume by current API per-token rates over the same amortization period.
- Compare the two totals, and re-run the model whenever usage or GPU pricing shifts materially.
Volume is not the only variable that matters
A pure cost model misses the cases where self-hosting is the right call regardless of the crossover math, because compliance or data sovereignty requirements rule out an external API entirely. In those cases the comparison is not cost versus cost, it is self-hosting versus not being allowed to use the workload at all, which changes the decision even at volumes where an API would technically be cheaper. Conversely, a low-volume, non-sensitive workload rarely justifies self-hosting on cost grounds alone, no matter how appealing full control sounds.
Frequently asked questions
Does the crossover point change as GPU prices fall?
Yes, GPU and hardware pricing shifts meaningfully over time, so a crossover calculated with 2026 pricing should be revisited whenever new hardware generations or supply conditions change the underlying cost, rather than treated as a fixed answer.
Should we count our own engineering time in the comparison?
Yes, staff time for GPU operations, inference tuning and ongoing maintenance is a real cost that belongs in the total cost of ownership, even though it is easy to leave out of a simple per-token comparison.
Is a hybrid approach cheaper than choosing one or the other?
Often yes, since routing high-volume or sensitive workloads to self-hosted infrastructure while keeping low-volume or general tasks on an API lets each workload sit on whichever side of the crossover actually favors it.
How often should this cost comparison be redone?
At least annually, and any time usage volume changes by more than roughly 30 to 50 percent or a new GPU generation shifts hardware pricing meaningfully, since both API rates and hardware costs move independently over time.
How Nanobase AI helps
Nanobase AI models total cost of ownership against actual or projected usage before recommending self-hosting over an API, using the client's own volume data rather than generic industry figures, and verifies current pricing on both sides since rates change frequently. This analysis connects directly to the disadvantages of self-hosting and the broader own GPUs versus cloud API cost guide. See /demo to walk through a live sizing example.
Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.