How long does fine-tuning take on a single H100?

On a single H100, a LoRA or QLoRA fine-tuning run over a typical instruction dataset of one thousand to five thousand examples on a 7B to 14B model usually completes in about one to six hours, depending on sequence length, number of epochs and batch size. Larger models push this up considerably, so a 70B model fine-tuned with QLoRA on a single H100 can take roughly one to two days for a similar size dataset, mainly because of slower forward and backward passes despite the reduced memory footprint. Full fine-tuning is not realistic on a single H100 for models above roughly ten billion parameters, since optimizer state memory alone exceeds the 80 GB of HBM3 available. Actual runtime also depends heavily on sequence length, since long-context examples multiply both compute and activation memory per step. Data loading, checkpointing and evaluation passes add overhead on top of raw training time, often ten to twenty percent in a well-configured pipeline. Nanobase AI, an NVIDIA Inception Program member, benchmarks expected training time against a client's actual dataset before scheduling GPU capacity.

The four variables that set total training time

Total fine-tuning time on a single H100 comes down to four multiplying factors, and understanding each separately explains why two projects with the same example count can take very different amounts of wall-clock time.

Sequence length is the variable most often underestimated, since its effect compounds with attention computation rather than scaling linearly like epoch count does.

A time estimate table by model size and method

These ranges assume typical instruction-tuning sequence lengths and standard batch sizes with gradient checkpointing; longer sequences or larger datasets push times toward the upper end or beyond. A 70B QLoRA run taking roughly a day is not a sign something is misconfigured; it reflects genuinely more compute per token at that scale.

Why sequence packing changes the picture significantly

Many training pipelines pad every example to a fixed maximum sequence length, which wastes compute on padding tokens that carry no training signal, especially when example lengths vary widely. Sequence packing concatenates multiple shorter examples into a single training sequence up to the maximum length, which can substantially improve GPU utilization and cut total training time compared to naive padding, particularly for datasets with a wide spread of example lengths such as short classification labels mixed with longer generation targets.

1. Check whether your training framework supports sequence packing natively, since most modern frameworks including Axolotl and TRL do.
2. Measure the distribution of example lengths in your dataset before choosing a maximum sequence length.
3. Enable packing and compare wall-clock time and GPU utilization against an unpacked baseline run.
4. Confirm loss masking still correctly isolates each packed example's response tokens, since packing can complicate masking if not implemented carefully.

Enabling sequence packing is often the single highest-leverage change for reducing training time without touching model size or dataset size at all.

Overhead beyond the raw training loop

The raw forward and backward pass time is not the whole story. Data loading, periodic checkpointing to save recovery points, and evaluation passes run at intervals during training all add wall-clock time on top of the core training computation, commonly adding 10 to 20 percent overhead in a well-configured pipeline, more if checkpointing frequency or evaluation set size is set aggressively high. Flash attention implementations, now standard in most current frameworks, meaningfully reduce the per-step cost of longer sequences and are worth confirming are enabled rather than assumed, since a misconfigured environment can silently fall back to a slower attention implementation.

Data loading, checkpointing, and flash attention configuration routinely account for more of the time gap between two runs than the raw model size difference does.

Frequently asked questions

Does a larger batch size always reduce total training time?

Only up to the point where GPU memory and compute are fully utilized; beyond that, a larger batch size mainly changes the training dynamics rather than wall-clock speed, and very large batches can sometimes need learning rate adjustment to train stably.

How much does using QLoRA instead of full-precision LoRA affect training time?

QLoRA is typically somewhat slower per step than full-precision LoRA due to the overhead of dequantizing 4-bit weights during computation, though this is usually outweighed by fitting a larger model or batch size on the same hardware.

Should we run a full training job overnight without checking early progress?

No, checking the loss curve within the first few hundred steps catches a misconfigured learning rate or data pipeline issue early, saving the cost of a full run that would have failed anyway.

Does context length during inference need to match the training sequence length?

Not exactly, but training predominantly on short sequences and then serving very long context at inference can produce degraded quality on the long-context portion the model rarely saw during training.

How Nanobase AI helps

Nanobase AI, an NVIDIA Inception Program member, benchmarks expected training time against a client's actual dataset, including sequence length distribution and packing configuration, before scheduling GPU capacity, so time and cost estimates reflect the real workload rather than generic published figures. See our solutions for GPU infrastructure sizing across fine-tuning projects.

Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.