What is the best fine-tuning dataset format: ChatML, ShareGPT or Alpaca?

There is no single best format, since the right choice depends on the target model's expected chat template and whether your task is single-turn or multi-turn. ChatML, which wraps each turn with explicit role tags for system, user and assistant, is the native format many open-weight chat models were themselves instruction-tuned with, so using it for further fine-tuning keeps the model consistent with how it expects conversations structured at inference time. ShareGPT format, structured as a list of conversation turns, is popular for multi-turn conversational data and is well supported by frameworks like Axolotl, making it a practical choice when your training examples involve back-and-forth dialogue rather than single exchanges. Alpaca's simpler instruction, input and output structure is easiest to generate and works well for single-turn tasks like classification, extraction or short-answer generation, but it is a poor fit for multi-turn conversation training. The most important rule regardless of format choice is matching whatever chat template the base model was originally trained with, since a mismatch here quietly degrades quality. Nanobase AI converts client data into whichever format the target base model expects before training begins.

Match the format to the model's own chat template, first

The single most important selection factor is not which format is most popular in tutorials, but which chat template your target base model was itself instruction-tuned with, since fine-tuning further on a mismatched format can degrade the model's ability to follow its expected conversational structure at inference time. Most current open-weight chat models expose their native chat template through their tokenizer configuration, and the safest default is training data structured to match that template exactly, converting from whichever raw format your source data happens to be in. Framework compatibility and tutorial popularity are secondary; the model's own expected template should decide the format, and every major training framework can convert between formats before training starts.

Structural comparison

The structural difference between these three formats comes down almost entirely to how many conversation turns each one is built to represent.

A minimal ChatML-style example looks like this:

<|im_start|>system
You are a support assistant for Acme billing questions.
<|im_end|>
<|im_start|>user
Why was I charged twice this month?
<|im_end|>
<|im_start|>assistant
Looking at your account, the second charge was a prorated upgrade fee...
<|im_end|>

When each format is the clearly better fit

Alpaca's flat structure is genuinely simpler to generate and review for single-turn tasks like classification, extraction or one-shot question answering, where there is no conversation history to represent and the extra structure of ChatML or ShareGPT adds no value. ShareGPT's explicit conversation-array structure is the more natural fit once training data includes multi-turn exchanges with prior context the model needs to reference, since it represents the full turn sequence directly rather than requiring the target model's template to be reconstructed from a flatter representation. ChatML is less a dataset format choice and more a wire-format decision usually made for you by the target model's tokenizer configuration, and most frameworks will convert ShareGPT or Alpaca-structured data into a model's native ChatML-style template automatically during preprocessing.

A conversion-first workflow

Collecting data in whatever format is easiest to audit, then converting at training time, avoids locking your data pipeline to one specific model's template.

1. Identify your target base model's native chat template from its tokenizer configuration before choosing a working format for data collection.
2. Collect and store raw training data in whichever structure is easiest to review and audit, which is often ShareGPT-style for multi-turn or Alpaca-style for single-turn, regardless of the model's native template.
3. Use your training framework's built-in format conversion (Axolotl, Unsloth and TRL all support this) to transform the working format into the model's native template at training time.
4. Spot-check the converted output for a sample of examples to confirm role tags, special tokens and turn boundaries rendered correctly, since silent template mismatches are a common source of degraded fine-tuning results that are hard to diagnose after the fact.
5. Keep the same conversion pipeline in place for future retraining rounds, so switching base models later only requires updating one template mapping rather than reformatting the entire dataset.

Framework support matters more than format purity

Axolotl, Unsloth and Hugging Face TRL each support all three formats through configuration rather than requiring separate codebases, which means format choice should be driven by convenience and clarity for your specific task rather than by framework compatibility concerns. This connects directly to broader dataset construction from company data, where the underlying content quality matters far more to final model performance than which of these three structurally similar formats it happens to be stored in.

Frequently asked questions

Does the dataset format affect final model quality?

The format itself has minimal direct effect on quality as long as it correctly represents your training examples and is converted properly to the target model's native template; quality differences come from data content and diversity, not from the choice between these three structurally similar formats.

Can we mix formats within one training run?

Technically most frameworks can ingest multiple formats if each is converted to the same internal representation before training, but it is simpler and less error-prone to standardize on one working format for data collection and let the framework handle template conversion at training time.

Which format is best for multi-turn customer support conversations?

ShareGPT's conversation-array structure is the most natural fit for multi-turn support data, since it explicitly represents the full back-and-forth sequence, including cases where the assistant must reference something said several turns earlier in the same conversation thread rather than treating each turn in isolation.

How Nanobase AI helps

Nanobase AI standardizes dataset formatting as part of every fine-tuning engagement, mapping company data into the target model's native chat template with automated validation, so format mismatches never become a silent cause of underperformance. This is one piece of our complete fine-tuning delivery process.

Ready to discuss your project? Contact Nanobase AI or email hello@bumu.tech.