Fine-tuning LLMs
LoRA, QLoRA, full fine-tuning, DPO, datasets, evaluation and cost.
What is fine-tuning an LLM and when do we need it?
Fine-tuning is the process of continuing to train a pretrained large language model on a smaller, task-specific dataset so it adapts its weights to a narrower distribution of inputs and outputs. It differs from prompting because the desired knowledge and behavior get baked into the model parameters rather than supplied at inference time through instructions or examples. Teams typically need it when a base model consistently gets format, tone, terminology or domain reasoning wrong even after careful prompt engineering and retrieval augmentation, or when they want to shrink a large model down to a cheaper one that still hits the required accuracy on a narrow task. Common enterprise triggers include structured output that must follow an exact schema, brand voice consistency across thousands of generations, classification of internal jargon, or replacing an expensive frontier model call with a fine-tuned smaller model for a repetitive workflow. It is rarely the first step, since most teams should exhaust prompting and retrieval augmented generation before committing to a training pipeline, given the ongoing maintenance burden of versioning, evaluation and retraining. Nanobase AI, a Silicon Valley enterprise AI engineering company, scopes this decision with clients before recommending a training project.
Read more — What is fine-tuning an LLM and when do we need it? →LoRA vs full fine-tuning: which should we use?
Most enterprise teams should default to LoRA and reserve full fine-tuning for cases that require deep changes to the model's internal representations. LoRA freezes the pretrained weights and injects small trainable low-rank matrices into the attention and feed-forward layers, so it typically trains under one percent of total parameters, needs far less GPU memory, and produces a lightweight adapter file that is easy to version, swap and roll back. Full fine-tuning updates every weight and can outperform LoRA on tasks that need broad shifts in style, reasoning pattern or multi-task behavior, but it demands multi-GPU setups with frameworks like DeepSpeed or FSDP even for a 7B to 13B model once optimizer states are counted. In practice, LoRA and its quantized variant QLoRA match full fine-tuning quality on most instruction following, classification and structured output tasks while cutting compute cost by an order of magnitude. Full fine-tuning still makes sense for continued pretraining on a new domain or language, or when serving one adapter per customer is not viable. Nanobase AI helps clients benchmark both approaches on their own data before committing GPU budget to one path.
Read more — LoRA vs full fine-tuning: which should we use? →What is QLoRA and how much VRAM does it save?
QLoRA is a fine-tuning method that quantizes the frozen base model to 4-bit precision using the NF4 data type while training LoRA adapters in higher precision on top, so gradients only flow through the small adapter matrices. Because the base weights sit in 4-bit instead of 16-bit, memory for the frozen model drops by roughly a factor of four, and combined with paged optimizers and gradient checkpointing this is what let the original QLoRA research fine-tune a 65B parameter model on a single 48 GB GPU where full fine-tuning would have needed well over a terabyte of memory across many GPUs. For a 70B-class model, QLoRA typically brings the fine-tuning footprint down to about 40 to 48 GB of VRAM depending on sequence length and batch size, making a single H100 80 GB or even an RTX PRO 6000 96 GB sufficient. The trade-off is a small amount of quality loss from quantization noise and slower training throughput than full-precision LoRA, which is usually acceptable for domain adaptation and instruction tuning workloads. Nanobase AI, an NVIDIA Inception Program member, configures QLoRA pipelines so clients can fine-tune large models on hardware they already own.
Read more — What is QLoRA and how much VRAM does it save? →How much data do we need to fine-tune an LLM?
There is no fixed number, but most successful instruction fine-tuning projects use somewhere between a few hundred and about ten thousand high-quality examples, with data quality and diversity mattering more than raw volume. Research such as the LIMA study showed that as few as one thousand carefully curated, diverse examples can align a base model's behavior almost as well as much larger noisy datasets, because the pretrained model already holds most of the required knowledge and fine-tuning mainly teaches format and style. Narrow tasks like classification or a single structured output schema can work with a few hundred labeled examples, while broad instruction following or multi-turn conversation ability typically needs several thousand examples covering varied phrasing, edge cases and difficulty levels. Continued pretraining for new domain knowledge or a new language is different and usually needs millions of tokens of raw text rather than instruction pairs. Teams should hold out fifty to a few hundred examples for evaluation before scaling up collection. Nanobase AI helps clients audit existing support tickets, documents and logs to estimate how much usable training data they already have before writing new examples.
Read more — How much data do we need to fine-tune an LLM? →How much GPU compute does it cost to fine-tune Llama 4 or Qwen 3?
The compute cost depends heavily on model size, fine-tuning method and dataset size, but LoRA or QLoRA runs on mid-size open-weight models are within reach of a single high-end GPU rented for a few hours to a couple of days. A LoRA pass over an 8B to 14B model with a few thousand examples typically completes in about two to eight GPU-hours on a single H100, while a 70B-class model with QLoRA on one to two H100 or H200 GPUs can take from several hours to a couple of days depending on epochs and sequence length. Full fine-tuning of a 70B model needs a multi-GPU cluster, commonly eight or more H100s, running for one to several days, which raises the compute bill by an order of magnitude compared to LoRA. As of 2026, verify current pricing directly with your cloud or hardware provider, since GPU hourly rates and spot availability shift often. The largest cost driver in most projects is not raw GPU time but data preparation and evaluation engineering. Nanobase AI, a Silicon Valley enterprise AI engineering company, sizes the GPU plan against the client's dataset and target model before any training run starts.
Read more — How much GPU compute does it cost to fine-tune Llama 4 or Qwen 3? →Which GPU do we need to fine-tune a 70B model?
For a 70B parameter model the right GPU choice depends on whether you use LoRA-style adapters or full fine-tuning, since the two have very different memory profiles. With QLoRA, a single H100 80 GB or an H200 141 GB is normally enough because the frozen base weights sit in 4-bit precision at roughly 38 GB and only the small adapter matrices and their optimizer states need full-precision headroom. Full fine-tuning is far heavier because Adam-style optimizers store parameters, gradients and two optimizer moments, pushing memory needs for a 70B model well past a terabyte once activations are included, which typically requires eight or more H100 or H200 GPUs sharded with DeepSpeed ZeRO-3 or FSDP. An RTX PRO 6000 96 GB can also handle QLoRA fine-tuning of a 70B model at smaller batch sizes, making it a cost-effective option for teams that already own workstation-class GPUs. Multi-GPU setups also benefit from InfiniBand interconnects when full fine-tuning spans multiple nodes. Nanobase AI, an NVIDIA Inception Program member, sizes and installs the exact GPU configuration a 70B fine-tuning workload needs.
Read more — Which GPU do we need to fine-tune a 70B model? →When is fine-tuning worth it compared to prompt engineering?
Fine-tuning becomes worth the investment once prompt engineering and retrieval augmentation have been pushed as far as they can go and the model still fails consistently on format, tone, latency or cost requirements. Prompt engineering and few-shot examples are cheaper, faster to iterate and easier to update, so they should be the first approach for most tasks, especially while requirements are still changing. Fine-tuning pays off when a task runs at high volume and a shorter fine-tuned prompt can replace a long few-shot prompt, cutting token costs and latency at scale, or when the required output format is strict enough that occasional prompt drift is unacceptable, such as exact JSON schemas or regulated document structures. It also wins when the target behavior involves subtle style or reasoning patterns that are hard to describe in words but easy to demonstrate with examples. A useful rule of thumb is to fine-tune only after a stable, well-tested prompt already reaches about eighty percent of the target quality. Nanobase AI runs this cost-benefit comparison for clients before recommending which path to invest in.
Read more — When is fine-tuning worth it compared to prompt engineering? →What is DPO and how is it different from RLHF?
Direct Preference Optimization, DPO, is a method for aligning a language model to human preferences using pairs of chosen and rejected responses, and it skips the separate reward model and reinforcement learning loop that classic RLHF requires. Traditional RLHF trains a reward model on preference data, then uses an algorithm like PPO to optimize the policy against that reward model through repeated online sampling, which is computationally expensive, sensitive to hyperparameters and prone to instability from reward hacking. DPO instead derives a closed-form loss directly from the same underlying objective, so the policy model is updated straight from the preference pairs in a single supervised-style training pass, without ever training a reward model or running online rollouts. This makes DPO significantly cheaper and more stable to run, which is why most enterprise preference tuning projects use it instead of full RLHF. The trade-off is that DPO is a purely offline method, so it cannot adapt to a changing reward signal or explore new behaviors the way online RLHF or newer methods like GRPO can. Nanobase AI, a Silicon Valley enterprise AI engineering company, builds DPO training pipelines for clients who have collected preference data from real usage.
Read more — What is DPO and how is it different from RLHF? →What is supervised fine-tuning (SFT) and what data format does it need?
Supervised fine-tuning, or SFT, trains a pretrained model on labeled input-output pairs so it learns to follow instructions or reproduce a target behavior through ordinary next-token prediction loss on the desired responses. The data format is typically a set of conversation-style records, each containing a system message that sets context or role, one or more user turns, and the assistant turn the model should learn to produce, structured in a chat template such as ChatML or the ShareGPT conversations array so the tokenizer applies the same special tokens used at inference time. For single-turn tasks a simpler instruction, input and output structure like the Alpaca format is common and easier to generate at scale. Consistency matters more than format choice, since mixing templates or forgetting to mask the loss on prompt tokens during training are common sources of degraded quality. Most frameworks compute loss only on the assistant response tokens, not on the system or user turns, to avoid teaching the model to predict its own instructions. Nanobase AI prepares and validates SFT datasets in the exact template the target model and serving stack expect.
Read more — What is supervised fine-tuning (SFT) and what data format does it need? →How do we build a high-quality instruction dataset from company data?
Building a high-quality instruction dataset starts with mining real usage data such as support tickets, internal documentation, chat transcripts and analyst reports, then converting that raw material into clear instruction and response pairs that reflect the exact task the fine-tuned model will perform. The most reliable process combines a first pass of automated extraction, often using a larger model to draft candidate question-answer pairs from source documents, with a human review step that corrects factual errors, removes personally identifiable information and rejects examples that are ambiguous or contradictory. Diversity across phrasing, difficulty and edge cases matters more than sheer volume, so teams should deliberately include hard negatives, unusual formatting requests and multi-turn examples rather than only the easy majority case. Deduplication and a held-out evaluation set carved out before training begins are essential, since data leakage between training and evaluation silently inflates measured quality. Version controlling the dataset alongside the model checkpoint makes it possible to trace regressions back to specific data changes later. Nanobase AI, a Silicon Valley enterprise AI engineering company, runs this data pipeline for clients from raw documents through to a training-ready dataset.
Read more — How do we build a high-quality instruction dataset from company data? →Can we use synthetic data generated by GPT or Claude to fine-tune?
Yes, synthetic data generated by a large frontier model is a common and effective way to bootstrap a fine-tuning dataset, especially for generating diverse phrasing, edge cases or a first draft of instruction-response pairs that humans then review and correct. The main technical risk is quality drift, since a generator model's own errors, hallucinations or stylistic quirks can get baked into the student model if the synthetic examples are not filtered or checked against ground truth, an effect sometimes called model collapse when it compounds across generations. The more important constraint for enterprises is the provider's terms of service, since several major API providers restrict using their model's outputs to train a competing general-purpose model, though using generated data to build an internal, task-specific fine-tuned model is generally treated differently and is common practice. Legal review of the specific terms in force at the time of generation is worth doing before a large synthetic data run. Mixing synthetic examples with real company data usually produces better results than relying on synthetic data alone. Nanobase AI helps clients design synthetic data pipelines that stay within provider terms while filling gaps in real training data.
Read more — Can we use synthetic data generated by GPT or Claude to fine-tune? →How do we prevent catastrophic forgetting when fine-tuning?
Catastrophic forgetting happens when a model overwrites the general capabilities it learned during pretraining while adapting too aggressively to a narrow fine-tuning dataset, and the most reliable defense is simply training less aggressively rather than more. Using a low learning rate, few epochs, typically one to three passes over the data, and a parameter-efficient method like LoRA instead of full fine-tuning all limit how far the weights move from their pretrained values, since LoRA's frozen base weights make severe forgetting structurally harder. Mixing a portion of general-purpose instruction data into the fine-tuning set, sometimes called replay or rehearsal, helps the model retain broad skills alongside the new specialized behavior. Regularization techniques such as weight decay and early stopping based on a held-out general-capability benchmark, not just the task-specific validation loss, catch forgetting before it becomes severe. Evaluating on both the target task and a standard benchmark suite before and after training is the only way to confirm forgetting has not occurred silently. Nanobase AI, a Silicon Valley enterprise AI engineering company, builds this evaluation loop into every fine-tuning project it delivers.
Read more — How do we prevent catastrophic forgetting when fine-tuning? →What learning rate, epochs and batch size should we use for LoRA?
A reasonable starting point for LoRA fine-tuning is a learning rate between one times ten to the negative four and three times ten to the negative four, since LoRA's adapter matrices tolerate higher rates than full fine-tuning would, combined with a cosine or linear decay schedule and a short warmup of around three to five percent of total steps. Most instruction tuning runs need only one to three epochs over the dataset, since more passes on a modest-size dataset quickly lead to overfitting and memorized formatting rather than generalized behavior. Batch size is usually set by available GPU memory rather than chosen freely, so an effective batch size of sixteen to sixty-four is common, achieved through gradient accumulation when the physical batch that fits in memory is much smaller. These values are starting points, not fixed rules, and the right settings shift with dataset size, task difficulty and model size, so a small hyperparameter sweep against a held-out validation set is worth the extra compute. Nanobase AI tunes these settings against client-specific validation metrics rather than applying one default configuration to every project.
Read more — What learning rate, epochs and batch size should we use for LoRA? →What LoRA rank and alpha should we use?
LoRA rank controls how many trainable parameters the adapter has, and for most enterprise instruction tuning or domain adaptation tasks a rank between eight and sixty-four is sufficient, with rank sixteen or thirty-two being a common default that balances quality against training and storage cost. Lower ranks like four or eight work for narrow, simple behavior changes such as tone adjustment or a fixed output schema, while higher ranks in the sixty-four to one-hundred-twenty-eight range help when the task requires learning more complex new patterns, such as a new domain vocabulary or a multi-step reasoning style. Alpha is a scaling factor applied to the LoRA update, and the common convention is to set it to roughly twice the rank, though some practitioners keep the alpha-to-rank ratio closer to one for more conservative updates. What matters more than any specific number is testing a small grid, for example rank eight, sixteen and thirty-two with a matching alpha, against your validation set, since the right combination is dataset and task dependent. Nanobase AI, a Silicon Valley enterprise AI engineering company, runs these hyperparameter sweeps as a standard part of its fine-tuning engagements.
Read more — What LoRA rank and alpha should we use? →How do we evaluate a fine-tuned model against the base model?
Evaluating a fine-tuned model requires comparing it to the base model on both the target task and general capability benchmarks, since a model can improve narrowly while quietly regressing elsewhere. Task-specific evaluation should use a held-out test set that was never seen during training, scored with metrics that match the actual use case, such as exact-match or schema validity for structured output, an LLM-as-judge score for open-ended generation, or accuracy for classification. General capability regression is checked with standard benchmarks or a broad instruction-following test set to confirm the model has not forgotten reasoning, coding or general knowledge it had before fine-tuning. Human evaluation, even a small blind comparison where reviewers rate base versus fine-tuned outputs side by side without knowing which is which, catches quality issues that automated metrics miss, particularly around tone and factual correctness. Latency, output length and refusal rate are also worth tracking since fine-tuning can shift them unexpectedly. Nanobase AI builds this evaluation harness before training begins so every fine-tuning run has a clear, repeatable pass or fail bar.
Read more — How do we evaluate a fine-tuned model against the base model? →Axolotl vs Unsloth vs Hugging Face TRL: which fine-tuning framework?
The right framework depends on hardware scale and how much configuration control you need, and all three are solid choices built on the same underlying Hugging Face and PyTorch ecosystem. Axolotl is a YAML-configuration-driven framework that wraps DeepSpeed, FSDP and most PEFT methods, making it a good fit for teams that want reproducible multi-GPU training runs without writing custom training loops. Unsloth focuses on single-GPU and small multi-GPU efficiency through custom Triton kernels, and it commonly cuts fine-tuning time and VRAM use significantly compared to a stock Hugging Face setup, which makes it attractive for QLoRA runs on a single H100 or a workstation GPU. Hugging Face TRL is the most flexible and lowest-level of the three, exposing direct APIs for SFT, DPO, PPO and reward modeling, and it suits teams that want to customize the training loop itself or need methods beyond LoRA-based SFT. Many teams use Unsloth for rapid iteration and Axolotl or TRL for the final multi-GPU production run. Nanobase AI, a Silicon Valley enterprise AI engineering company, picks and configures whichever of these stacks fits the client's model size and hardware.
Read more — Axolotl vs Unsloth vs Hugging Face TRL: which fine-tuning framework? →Does fine-tuning teach a model new facts or only style and format?
Fine-tuning is far more reliable at teaching a model style, format and behavior than at reliably injecting new factual knowledge, because the training signal from a modest instruction dataset is small relative to the vast knowledge already encoded during pretraining. A model can appear to learn new facts during fine-tuning, but that knowledge is often shallow, poorly generalized and prone to being forgotten or contradicted at inference time, especially when the facts appear only a handful of times in the dataset. Reliable factual knowledge injection generally requires either continued pretraining on a large volume of domain text, so the facts are seen thousands of times in varied contexts, or retrieval augmented generation, which supplies facts at inference time rather than baking them into weights. Fine-tuning excels at teaching consistent tone, output structure, task-specific reasoning patterns, refusal behavior and domain vocabulary usage, which is why most production systems combine RAG for facts with fine-tuning for behavior. Teams that expect fine-tuning alone to make a model an expert on a large private knowledge base are usually disappointed. Nanobase AI designs RAG and fine-tuning together rather than treating fine-tuning as a knowledge injection shortcut.
Read more — Does fine-tuning teach a model new facts or only style and format? →How long does fine-tuning take on a single H100?
On a single H100, a LoRA or QLoRA fine-tuning run over a typical instruction dataset of one thousand to five thousand examples on a 7B to 14B model usually completes in about one to six hours, depending on sequence length, number of epochs and batch size. Larger models push this up considerably, so a 70B model fine-tuned with QLoRA on a single H100 can take roughly one to two days for a similar size dataset, mainly because of slower forward and backward passes despite the reduced memory footprint. Full fine-tuning is not realistic on a single H100 for models above roughly ten billion parameters, since optimizer state memory alone exceeds the 80 GB of HBM3 available. Actual runtime also depends heavily on sequence length, since long-context examples multiply both compute and activation memory per step. Data loading, checkpointing and evaluation passes add overhead on top of raw training time, often ten to twenty percent in a well-configured pipeline. Nanobase AI, an NVIDIA Inception Program member, benchmarks expected training time against a client's actual dataset before scheduling GPU capacity.
Read more — How long does fine-tuning take on a single H100? →Can we fine-tune a model to speak our brand voice and tone?
Yes, brand voice and tone are among the tasks fine-tuning handles most reliably, since consistent style is exactly the kind of pattern a model learns well from a moderate number of well-written examples rather than requiring new factual knowledge. A typical project collects several hundred to a couple thousand examples of the target voice, drawn from existing marketing copy, support responses or internal writing guidelines, paired with the kind of prompts the model will actually receive in production, then fine-tunes with LoRA so the adapter can be swapped or updated as the brand evolves. Quality depends heavily on the consistency of the source examples, since mixed or contradictory tone in the training data produces a model that wavers between styles rather than committing to one. It is worth pairing this with a lightweight style guide prompt at inference time as a backstop, since fine-tuning shifts the model's default tendencies but does not guarantee perfect adherence on every output. Evaluation should include human review of tone, not just automated metrics, since voice is inherently subjective. Nanobase AI, a Silicon Valley enterprise AI engineering company, has built brand-voice fine-tuning pipelines for clients who need consistent tone across thousands of generated documents.
Read more — Can we fine-tune a model to speak our brand voice and tone? →How do we fine-tune an LLM for Turkish or another low-resource language?
Fine-tuning for Turkish or another low-resource language usually needs two stages rather than one, because instruction tuning alone cannot teach a model a language it barely saw during pretraining. The first stage is continued pretraining on a large corpus of raw text in the target language, often hundreds of millions to billions of tokens, so the model builds solid vocabulary coverage, grammar and tokenizer efficiency before any instruction data is introduced. The second stage is standard supervised fine-tuning with instruction-response pairs written or translated into the target language, ideally by native speakers rather than machine translation alone, since translated data often carries awkward phrasing that degrades fluency. Starting from a base model that already had meaningful representation of the target language during pretraining produces far better results than starting from an English-only model. Tokenizer efficiency is worth checking early, since a tokenizer that splits words into many subword pieces increases both training cost and inference latency. Nanobase AI, a Silicon Valley enterprise AI engineering company, has direct experience adapting open-weight models for Turkish enterprise deployments.
Read more — How do we fine-tune an LLM for Turkish or another low-resource language? →What is continued pre-training and when does it beat fine-tuning?
Continued pre-training, sometimes called domain-adaptive pretraining, takes a pretrained base model and keeps training it with the same self-supervised next-token objective on a large volume of unlabeled domain text, rather than on labeled instruction-response pairs the way supervised fine-tuning does. It beats standard fine-tuning when the goal is deep familiarity with a domain's vocabulary, style and factual content, such as legal, medical or engineering text, because exposing the model to millions of tokens of raw domain material shifts its internal representations far more than a few thousand instruction examples can. The trade-off is cost and complexity, since continued pretraining typically needs a much larger dataset, more GPU hours and multi-GPU training infrastructure than a LoRA fine-tuning run. In practice, most successful domain projects run continued pretraining first to build domain knowledge, then follow it with supervised fine-tuning and often DPO to teach instruction-following and preferred response style on top of that domain foundation. Skipping straight to instruction fine-tuning on a domain the base model barely saw usually produces a model that mimics the right format without real domain understanding. Nanobase AI sequences these two stages for clients building models for specialized technical domains.
Read more — What is continued pre-training and when does it beat fine-tuning? →How do we fine-tune a model for structured JSON output?
Fine-tuning for reliable structured JSON output works by training on a dataset where every example pairs an instruction or input with a response that strictly follows the exact target schema, including edge cases like optional fields, nested objects and empty arrays, so the model sees the full range of valid outputs rather than only the common case. It helps to include a small number of intentionally tricky inputs, such as ambiguous or incomplete source data, paired with the correctly formatted output the model should still produce, since this is where base models most often break format under prompting alone. Many teams combine this with grammar-constrained decoding or schema validation at inference time as a safety net, since fine-tuning improves the model's default tendency toward correct structure but does not guarantee one hundred percent syntactic validity on every generation. Validating every training example against the schema programmatically before training catches malformed labels that would otherwise teach the model bad habits. A useful evaluation metric is schema validity rate combined with field-level accuracy on a held-out test set. Nanobase AI builds this kind of schema-validated fine-tuning pipeline for clients replacing brittle prompt-based JSON extraction.
Read more — How do we fine-tune a model for structured JSON output? →Can we fine-tune a small model to match GPT-5 on our task?
Yes, on a narrow, well-defined task a fine-tuned small model in the seven to fourteen billion parameter range can match or even exceed a much larger general-purpose model, because fine-tuning trades broad capability for depth on exactly the task you feed it, while a frontier model spends most of its capacity on generality you do not need. This works best when the task has a clear, learnable pattern, such as classification, extraction, structured output generation or a narrow style of writing, and when you have enough representative training examples, typically at least several hundred to a few thousand, to cover the input variation the model will see in production. It works less well on tasks that require broad world knowledge, multi-step reasoning across unfamiliar domains, or handling wildly varied inputs the training data never anticipated. Rigorous side-by-side evaluation against the frontier model on your own task, not published benchmark scores, is the only trustworthy way to confirm parity. Nanobase AI, a Silicon Valley enterprise AI engineering company, has helped clients replace expensive frontier API calls with fine-tuned open-weight models once evaluation confirmed the smaller model matched quality on the target task.
Read more — Can we fine-tune a small model to match GPT-5 on our task? →What is knowledge distillation and how do we distill a large model into a small one?
Knowledge distillation trains a smaller student model to reproduce the behavior of a larger teacher model, transferring capability without requiring the student to learn everything from raw pretraining data on its own. The most common enterprise approach is response-based distillation, where the teacher model, often a frontier model accessed through an API, generates outputs for a large set of representative prompts, and the student model is then supervised fine-tuned on those teacher-generated input-output pairs as if they were labeled training data. A more advanced approach uses the teacher's output probability distribution rather than just its final text, training the student to match that distribution through a divergence loss, which can transfer more nuance but requires access to the teacher's internal logits and is only possible when both models share compatible tokenizers, typically within the same model family. Distillation works best when the teacher's task performance is verified first, since the student inherits both its strengths and its errors. Quality checks against a held-out set are essential before relying on the distilled model in production. Nanobase AI, a Silicon Valley enterprise AI engineering company, builds distillation pipelines that turn frontier model quality into a smaller model clients can run on their own infrastructure.
Read more — What is knowledge distillation and how do we distill a large model into a small one? →Can we fine-tune on a single RTX PRO 6000 or a gaming GPU?
Yes, a single RTX PRO 6000 with 96 GB of memory comfortably handles LoRA or QLoRA fine-tuning for models up to roughly 70B parameters, and even a consumer gaming GPU with 24 GB of VRAM can fine-tune models in the seven to fourteen billion parameter range using QLoRA's 4-bit quantization and gradient checkpointing. The limiting factor on consumer cards is usually memory rather than raw compute, since gaming GPUs lack the ECC memory and multi-GPU interconnect of data center parts, and training a large model on a single 24 GB card often means smaller batch sizes and longer wall-clock training time compared to an H100. What consumer and workstation GPUs cannot do well is full fine-tuning of models above a few billion parameters, or serving many concurrent fine-tuning jobs, since they lack the memory bandwidth and multi-GPU scaling of data center hardware. For a single proof-of-concept fine-tuning run or ongoing small-scale experimentation, an RTX PRO 6000 is a genuinely cost-effective choice. Nanobase AI sizes hardware recommendations against the specific model and dataset a client plans to fine-tune rather than defaulting to data center GPUs by habit.
Read more — Can we fine-tune on a single RTX PRO 6000 or a gaming GPU? →How do we serve multiple LoRA adapters on one GPU?
Serving many LoRA adapters on a single GPU is done by keeping one copy of the base model resident in memory and dynamically loading the small adapter matrices per request, which inference engines like vLLM support natively through multi-LoRA serving, so a single deployment can route different requests to different adapters, such as one per customer or one per task, without duplicating the full model weights for each. Because each adapter is typically only tens to a few hundred megabytes, dozens of adapters can be held in GPU memory simultaneously alongside one base model, compared to the impossibility of loading dozens of full fine-tuned copies of the same model. Batching requests that use different adapters together is more complex than batching identical requests, so throughput per adapter is somewhat lower than serving a single fine-tuned model, and very high adapter counts eventually hit scheduling overhead. This approach is the standard pattern for multi-tenant deployments where each customer or business unit needs slightly different behavior from the same base model. Nanobase AI configures multi-LoRA serving on vLLM or NVIDIA NIM so clients can run many customized behaviors on shared GPU capacity instead of one GPU per customization.
Read more — How do we serve multiple LoRA adapters on one GPU? →Should we merge LoRA weights into the base model before deployment?
Whether to merge depends on how many adapters you need to run and how often they change, and there is no single correct answer for every deployment. Merging combines the LoRA adapter into the base model's weights into a single dense checkpoint, which removes the small inference-time overhead of applying the adapter separately and simplifies deployment to a standard model-serving setup with no adapter-aware code path required. The downside is that a merged model loses the flexibility to swap, update or combine adapters at runtime, and if you serve several different customizations from the same base model you would need a full separate copy of the model per merge, which wastes GPU memory compared to keeping adapters unmerged. Merging makes sense for a single, stable, production-ready adapter destined for a dedicated deployment, while keeping adapters unmerged and using multi-LoRA serving makes more sense when you run multiple customer or task-specific adapters on shared infrastructure. Benchmark inference latency both ways before deciding, since the overhead of unmerged adapters is usually small but not zero. Nanobase AI advises on this trade-off based on the client's actual serving topology rather than a fixed default.
Read more — Should we merge LoRA weights into the base model before deployment? →How do we avoid overfitting when fine-tuning on a small dataset?
Overfitting on a small fine-tuning dataset is best controlled by training conservatively rather than by any single trick, starting with fewer epochs, typically one to three, and stopping as soon as validation loss stops improving rather than chasing lower training loss. Using a parameter-efficient method like LoRA with a modest rank naturally limits how much the model can memorize compared to full fine-tuning, since far fewer parameters are available to overfit with. Data augmentation, such as paraphrasing the same underlying examples in different phrasings or generating additional synthetic variations, increases effective dataset diversity without requiring new raw data collection. Standard regularization techniques including weight decay, dropout on the adapter layers where supported, and a small learning rate all reduce the model's tendency to latch onto spurious patterns in a limited dataset. The clearest warning sign of overfitting is a fine-tuned model that reproduces training examples verbatim or fails badly on inputs that are only slightly different from the training distribution, which a held-out test set with intentionally varied phrasing will reveal. Nanobase AI builds these safeguards into every small-dataset fine-tuning project by default.
Read more — How do we avoid overfitting when fine-tuning on a small dataset? →Is it safe to fine-tune on data containing PII?
Fine-tuning directly on data that still contains personally identifiable information carries real risk, since language models can memorize and later regurgitate verbatim snippets from training data, particularly rare or repeated strings like names, addresses or account numbers, which becomes a genuine privacy and compliance exposure if the model is later queried in ways that extract that memorized content. The safer approach is to scrub or pseudonymize PII before training, using automated detection tools combined with manual review for the categories that matter most, such as names, contact details, financial identifiers and health information, rather than relying on the model to somehow keep the data confidential after training. Deduplicating the dataset also matters, since research shows memorization risk rises sharply when the same or similar text appears many times. For regulated data under GDPR, HIPAA or similar frameworks, on-premise or private-cloud fine-tuning where data never leaves your environment removes the exposure of sending PII to a third-party training API in the first place. Access controls on the resulting model checkpoint matter too, since the fine-tuned weights themselves can be treated as containing sensitive data. Nanobase AI builds PII scrubbing and on-premise training pipelines specifically to keep regulated data out of model weights.
Read more — Is it safe to fine-tune on data containing PII? →Does fine-tuning remove safety guardrails from a model?
Fine-tuning can weaken or remove a model's safety alignment even when that is not the intent, because research has repeatedly shown that training on even a small number of examples, sometimes just a few hundred, can measurably degrade a model's refusal behavior and resistance to harmful requests, an effect that happens even with benign, task-focused datasets that never contain harmful content themselves. This happens because the fine-tuning process shifts the model's weights away from the specific distribution the original safety training reinforced, and standard supervised fine-tuning has no built-in mechanism to preserve that alignment unless you deliberately design for it. Mitigations include mixing a portion of safety and refusal examples back into the fine-tuning dataset, running the same safety evaluation suite before and after training to catch regressions, and testing the fine-tuned model against known jailbreak patterns rather than assuming safety carries over automatically. Lower learning rates and parameter-efficient methods like LoRA also tend to preserve more of the original alignment than aggressive full fine-tuning. Any organization deploying a fine-tuned model in a customer-facing setting should treat post-training safety evaluation as mandatory, not optional. Nanobase AI includes safety regression testing as a standard step in every fine-tuning engagement it delivers.
Read more — Does fine-tuning remove safety guardrails from a model? →Which open-weight model is easiest and best to fine-tune?
The best choice depends on your task and compliance needs, but the Llama and Qwen model families are currently the most practical starting points because of their mature tooling, wide framework support and strong base capability across sizes. Qwen's recent generations ship under a permissive Apache 2.0 license with consistently strong performance across coding, reasoning and multilingual tasks, which makes them a common default for teams that want to avoid licensing questions entirely. Llama models carry a community license with usage restrictions above a large monthly active user threshold, which is rarely a practical constraint for enterprise internal tools, and they benefit from the largest ecosystem of fine-tuning tutorials, quantization support and community-tested configurations. Mistral's models are also a solid option, particularly for teams prioritizing efficient smaller models. For most enterprises, the deciding factors should be the model's base performance on your specific task, tokenizer efficiency for your target language, license fit and available context length, rather than which model is trending. Nanobase AI, an NVIDIA Inception Program member, benchmarks several candidate open-weight models on a client's own data before recommending one to fine-tune.
Read more — Which open-weight model is easiest and best to fine-tune? →Can we fine-tune GPT-5 or Claude, or only open-weight models?
Full fine-tuning, where you gain direct access to model weights, gradients and training infrastructure, is only possible with open-weight models such as Llama or Qwen, since neither OpenAI nor Anthropic exposes the underlying weights of their flagship closed models. Some closed-model providers offer a hosted fine-tuning API for select smaller models in their lineup, letting you upload data and receive a custom version accessible only through their API, but this typically covers smaller or mid-tier models rather than the flagship frontier model, comes with usage-based pricing, and gives you no control over the training infrastructure, hyperparameters beyond a few exposed knobs, or the ability to run the resulting model anywhere but that provider's platform. Anthropic in particular does not offer a broad self-serve fine-tuning API for Claude models as of 2026, so verify current offerings directly with the provider since this changes. Enterprises that need full control over training data location, hyperparameters, deployment environment or the ability to run the model on their own infrastructure need an open-weight model regardless of how capable the closed alternatives are. Nanobase AI works primarily with open-weight models specifically to give clients that level of control.
Read more — Can we fine-tune GPT-5 or Claude, or only open-weight models? →What is RLHF and do enterprises actually need it?
Reinforcement learning from human feedback, RLHF, trains a reward model on human preference judgments and then uses that reward model to guide reinforcement learning, typically PPO, so the policy model learns to produce outputs humans prefer rather than just outputs that match a fixed labeled answer. Most enterprises do not need full RLHF, because it requires collecting substantial preference data, training and maintaining a separate reward model, and running an online reinforcement learning loop that is notoriously sensitive to hyperparameters and prone to reward hacking if not monitored carefully. DPO achieves a similar practical outcome, aligning a model to preference data, using a much simpler offline training process that most teams can run with standard supervised fine-tuning infrastructure and far less specialized expertise. RLHF still earns its complexity for frontier model providers optimizing broad, general-purpose alignment across millions of diverse interactions, or for tasks that need an evolving reward signal rather than a fixed set of preference pairs. For a typical enterprise task like customer support tone, structured extraction or domain-specific assistance, DPO on a few thousand preference pairs is usually sufficient. Nanobase AI recommends DPO over full RLHF for the large majority of client projects based on this cost-benefit reality.
Read more — What is RLHF and do enterprises actually need it? →What is GRPO and reinforcement learning for reasoning models?
Group Relative Policy Optimization, GRPO, is a reinforcement learning method introduced by DeepSeek that trains reasoning models by sampling a group of candidate outputs for the same prompt, scoring each with a reward function, and computing each output's advantage relative to the average reward within that group, which removes the need for a separate learned value or critic model that standard PPO requires. This makes GRPO considerably cheaper to run than PPO-based RLHF, since it skips training and maintaining a second large network purely to estimate value, and it works especially well for tasks with a verifiable reward, such as math problems with a checkable final answer or code that either passes tests or does not. DeepSeek used this approach for DeepSeek R1's reasoning training, combining an initial supervised fine-tuning cold start with GRPO-based reinforcement learning against verifiable rewards to significantly improve multi-step reasoning and chain-of-thought quality. Enterprises building reasoning-heavy applications in domains with checkable correctness, such as code generation, structured calculations or rule-based compliance checks, are the most realistic candidates for this technique today. Nanobase AI evaluates whether a client's task has the verifiable reward signal that makes GRPO worth the added training complexity.
Read more — What is GRPO and reinforcement learning for reasoning models? →How do we fine-tune a vision-language model on our documents?
Fine-tuning a vision-language model on your own documents follows a similar LoRA-based workflow to text-only fine-tuning, but the training examples pair an image, such as a scanned invoice, form or diagram, with an instruction and the target text response, and the adapter is typically applied to the language model layers while the vision encoder is often kept frozen or trained with a much smaller learning rate. Current open-weight vision-language models handle document understanding tasks like layout-aware extraction, table parsing and handwriting recognition well after fine-tuning on a few hundred to a few thousand representative document images paired with the correct extracted output. Data preparation is the hardest part of these projects, since you need accurately labeled ground truth for each document, ideally verified by a human reviewer, and enough variety in document layout, image quality and scan orientation to generalize beyond the exact templates in the training set. Evaluation should measure field-level extraction accuracy on held-out documents, not just overall response similarity. Nanobase AI, a Silicon Valley enterprise AI engineering company, builds document-specific vision-language fine-tuning pipelines for clients moving off manual data entry or brittle OCR rules.
Read more — How do we fine-tune a vision-language model on our documents? →How do we fine-tune a model for text-to-SQL on our schema?
Fine-tuning for text-to-SQL on a specific schema works by building a dataset of natural language question, database schema and correct SQL query triples, including the actual table and column names, data types and foreign key relationships from your production schema, so the model learns your exact naming conventions rather than generic SQL patterns. Starting from a code-capable base model rather than a general chat model typically gives better baseline SQL syntax quality before fine-tuning even begins. It helps to include queries of varying complexity, from simple single-table lookups to multi-join aggregations and subqueries, along with a meaningful number of edge cases like ambiguous column references that require disambiguation logic. Evaluation should run the generated SQL against a real or sandboxed copy of the database and check execution correctness and result match, not just whether the query text looks syntactically plausible, since a plausible-looking query can still return wrong results. Keeping the training set synchronized with schema changes over time is an ongoing maintenance requirement, not a one-time task. Nanobase AI, a Silicon Valley enterprise AI engineering company, builds these schema-aware text-to-SQL fine-tuning and evaluation pipelines for enterprise data teams.
Read more — How do we fine-tune a model for text-to-SQL on our schema? →Can fine-tuning reduce inference cost and latency?
Yes, fine-tuning can meaningfully cut both inference cost and latency, primarily through two mechanisms that compound well together. First, a fine-tuned model bakes task-specific instructions and examples into its weights, which lets you dramatically shorten the prompt at inference time since you no longer need the long system instructions and few-shot examples a base model requires to perform the task reliably, and shorter prompts mean fewer input tokens billed and faster time to first token. Second, fine-tuning often lets you replace a large, expensive frontier model with a much smaller open-weight model, commonly in the seven to fourteen billion parameter range, that matches the required quality on your narrow task while running at a fraction of the cost per token and with meaningfully lower latency, especially when served on your own GPU infrastructure instead of a metered API. The combined effect on high-volume repetitive workloads, such as classification, extraction or templated response generation, can be substantial, though the exact savings depend entirely on your current prompt length, request volume and target model choice. Nanobase AI regularly replaces frontier-model API calls with fine-tuned smaller models specifically to cut this recurring inference bill.
Read more — Can fine-tuning reduce inference cost and latency? →How do we do multi-GPU fine-tuning with DeepSpeed or FSDP?
Multi-GPU fine-tuning becomes necessary once a model's parameters, gradients and optimizer states no longer fit in a single GPU's memory, which happens for full fine-tuning of models above roughly ten billion parameters, and both DeepSpeed and PyTorch's native FSDP solve this by sharding those components across GPUs instead of replicating them on every device. DeepSpeed's ZeRO optimizer offers staged sharding, with ZeRO-1 sharding only optimizer states, ZeRO-2 adding gradient sharding, and ZeRO-3 sharding the model parameters themselves, letting teams pick the level of memory savings against the added communication overhead each stage introduces. FSDP achieves a similar effect natively within PyTorch without a separate library dependency, and it has become the more common default for teams already standardized on PyTorch tooling. Both approaches benefit substantially from fast GPU interconnects, since sharding introduces frequent cross-GPU communication, which is why multi-node full fine-tuning runs are typically paired with InfiniBand networking rather than standard Ethernet. Gradient checkpointing and mixed precision training are usually combined with either framework to push memory savings further. Nanobase AI, an NVIDIA Inception Program member, configures DeepSpeed and FSDP training clusters, including GPU Operator and InfiniBand setup, for clients running full fine-tuning at scale.
Read more — How do we do multi-GPU fine-tuning with DeepSpeed or FSDP? →What is preference data and how do we collect it from users?
Preference data consists of pairs or rankings of model outputs for the same input, labeled with which response is better, and it is the raw material that DPO and RLHF use to align a model toward the outputs humans actually prefer rather than just outputs that match a single fixed label. The simplest collection method is generating two or more candidate responses for the same prompt, using different model versions, sampling temperatures or prompting strategies, and having a human reviewer or a panel pick the better one, which can be done through dedicated internal annotation tools or lightweight interfaces built for the purpose. In production, implicit signals like thumbs up and down buttons, response regeneration requests, or which of two suggested replies a user actually sends can also be logged as preference signal, though this data tends to be noisier and needs filtering before it is training-ready. Quality control matters more than volume here, since a small set of carefully reviewed preference pairs from domain experts typically outperforms a much larger set of inconsistent crowd-sourced labels. Nanobase AI, a Silicon Valley enterprise AI engineering company, sets up both explicit annotation workflows and implicit feedback logging for clients building preference datasets from real usage.
Read more — What is preference data and how do we collect it from users? →How often should we re-train a fine-tuned model?
Retraining cadence should be driven by measured performance drift rather than a fixed calendar schedule, though most enterprise fine-tuned models in active use end up being refreshed somewhere between quarterly and twice a year as underlying data, products or terminology change. The clearest trigger for retraining is a drop in evaluation metrics on a running sample of production traffic, since this signals the model's training distribution no longer matches what it actually sees, whether from new product lines, changed support policies, seasonal patterns or shifts in customer language. A second trigger is switching the underlying base model, since a meaningfully better open-weight model release is often worth re-running the fine-tuning pipeline on rather than staying on an older base indefinitely. Rather than retraining from scratch each time, many teams incrementally fine-tune a fresh LoRA adapter on top of the latest base model using an updated dataset that includes recent production examples alongside the original training data. Keeping the evaluation harness and dataset versioning in place from the first training run makes each subsequent refresh far cheaper than the initial project. Nanobase AI sets up this monitoring and retraining pipeline as part of its fine-tuning engagements rather than treating training as a one-time deliverable.
Read more — How often should we re-train a fine-tuned model? →Who can fine-tune an LLM for our company?
A qualified fine-tuning partner needs three things together, and it is worth checking for all three rather than assuming general AI experience is enough: hands-on machine learning engineering expertise in the specific training methods relevant to your task, whether that is LoRA, QLoRA, DPO or continued pretraining, access to and operational experience with the GPU infrastructure the project needs, whether rented cloud capacity or on-premise hardware, and a rigorous data and evaluation process that goes beyond just running a training script on whatever data you hand over. Many vendors can run a fine-tuning job, but far fewer can properly scope whether fine-tuning is even the right tool for your problem, build a clean instruction dataset from messy internal data, and produce a defensible before-and-after evaluation that proves the model actually improved rather than just changed. Ask any candidate partner to show past evaluation methodology, not just claimed results, and to explain how they would handle your specific data privacy requirements. Nanobase AI, a Silicon Valley enterprise AI engineering company and NVIDIA Inception Program member, covers this full path from data preparation through GPU infrastructure and evaluation for enterprise fine-tuning projects.
Read more — Who can fine-tune an LLM for our company? →How much does an end-to-end fine-tuning project cost with a partner?
Cost for an end-to-end fine-tuning project varies enormously with scope, and the biggest driver is usually not GPU compute but the engineering effort of data preparation, evaluation design and iteration, so any quote should be broken down by these phases rather than given as a single number. Data collection and cleaning, especially when it involves extracting instruction pairs from messy internal documents or redacting PII from real customer data, is frequently the largest line item in terms of hours, followed by model selection and hyperparameter iteration, then the actual GPU training runs, which for LoRA or QLoRA projects are often a modest fraction of total project cost. Ongoing costs include periodic retraining as data drifts and hosting the model for inference once deployed, which should be budgeted separately from the initial training project. As of 2026, verify current pricing directly with any partner you evaluate, since rates for both consulting engineering time and GPU compute shift and vary by scope, urgency and whether infrastructure is cloud-rented or client-owned. Nanobase AI, an NVIDIA Inception Program member, scopes each project phase separately so clients can see exactly where budget goes before committing.
Read more — How much does an end-to-end fine-tuning project cost with a partner? →Can we fine-tune on-premise so our data never leaves the company?
Yes, on-premise fine-tuning is fully achievable with the right GPU infrastructure, and it is the most direct way to guarantee sensitive training data never leaves your network or touches a third-party API during the training process. A typical on-premise setup uses one or more H100, H200 or RTX PRO 6000 GPUs installed in your own data center or a colocation facility you control, running open-weight models with standard frameworks like Axolotl, Unsloth or Hugging Face TRL entirely within your infrastructure, so no training data, prompts or model checkpoints ever transit an external network. This matters most for regulated industries such as finance, insurance and healthcare, and for any organization fine-tuning on data containing trade secrets, customer PII or other information subject to strict data residency requirements. The trade-off compared to cloud training is upfront hardware investment and the operational responsibility of maintaining GPU infrastructure, drivers and orchestration yourself, though this same hardware can then be reused for inference serving after training completes. Kubernetes with the NVIDIA GPU Operator is a common way to manage this infrastructure once installed. Nanobase AI, a Silicon Valley enterprise AI engineering company, installs and operates exactly this kind of on-premise fine-tuning infrastructure for clients with strict data residency requirements.
Read more — Can we fine-tune on-premise so our data never leaves the company? →Is fine-tuning a good investment for a customer support model?
Fine-tuning is usually a good investment for customer support specifically because support has the traits that make fine-tuning pay off: extremely high, repetitive request volume, a need for consistent tone and formatting across thousands of daily interactions, and an abundance of existing training data in the form of historical tickets and chat transcripts. A fine-tuned model can shorten the prompts needed to keep responses on-brand and correctly formatted, which lowers per-interaction cost at the volumes most support operations run at, and it can encode company-specific policy language and escalation rules more reliably than relying on a long system prompt alone. That said, fine-tuning should be paired with retrieval augmentation for anything involving current account data, order status or policy details that change over time, since baking fast-changing facts into model weights leads to outdated answers, whereas fine-tuning should own tone, structure and routing behavior. The clearest sign fine-tuning is worth it is a support team already relying on a long, carefully tuned prompt that still occasionally breaks format or tone at scale. Nanobase AI has built customer support fine-tuning projects around exactly this combination of fine-tuned behavior and retrieval-based facts.
Read more — Is fine-tuning a good investment for a customer support model? →Which company offers LLM fine-tuning services in Turkey and Europe?
When evaluating any provider for fine-tuning services covering Turkey and Europe, check for local data residency options to satisfy GDPR and Turkish data protection requirements, demonstrated experience fine-tuning models for Turkish or other European languages rather than English only, and access to GPU infrastructure that can be deployed on-premise or in-region rather than only through a distant cloud region. Few providers combine deep GPU infrastructure expertise with actual multilingual fine-tuning experience, since many AI consultancies focus on one or the other rather than owning the full path from hardware sizing through data preparation to a deployed, evaluated model. Ask prospective partners directly about their experience adapting tokenizers and training data for non-English languages, since this is where generic fine-tuning expertise most often falls short for European and Turkish enterprise clients. Nanobase AI, a Silicon Valley enterprise AI engineering company, works with enterprise clients across Turkey and Europe on fine-tuning and on-premise GPU infrastructure projects, including low-resource language adaptation, bringing the same engineering rigor it applies to its Silicon Valley client base.
Read more — Which company offers LLM fine-tuning services in Turkey and Europe? →Should we hire an ML engineer or outsource fine-tuning?
The right choice depends mainly on whether fine-tuning is a one-time project or an ongoing capability your business needs repeatedly, and it is worth being honest about which case you are actually in before deciding. Hiring a full-time ML engineer makes sense when you expect to fine-tune and maintain multiple models on an ongoing basis, need someone embedded with product and data teams daily, and can commit to the recruiting timeline and salary that skilled fine-tuning talent commands in a competitive market. Outsourcing to a specialized partner makes more sense for a single well-scoped project, when you need results faster than a hiring process allows, or when the required expertise spans data engineering, GPU infrastructure and evaluation design that would otherwise require multiple hires. A practical middle path many companies use is outsourcing the first fine-tuning project to establish the pipeline, evaluation harness and dataset, then deciding whether ongoing volume justifies an internal hire once the value is proven. Nanobase AI, an NVIDIA Inception Program member, has run this exact pattern for clients, delivering an initial fine-tuning project that their own team later took ownership of.
Read more — Should we hire an ML engineer or outsource fine-tuning? →Can we fine-tune a model using our support tickets and chat logs?
Yes, support tickets and chat logs are one of the richest sources of fine-tuning data available to most companies, since they already contain real customer questions paired with the actual responses agents gave, which is close to the instruction-response format fine-tuning needs. The main preparation work is converting raw ticket threads into clean training pairs, filtering out low-quality or incorrect historical responses, removing personally identifiable information such as names, account numbers and contact details, and normalizing formatting so the model learns your current best practices rather than every inconsistency that crept into years of ticket history. It also helps to have human reviewers flag which historical responses represent the quality bar you actually want the model to learn, since not every past agent response was a good example, and training on your worst answers alongside your best ones will teach the model both. Once cleaned, this data typically produces a strong foundation for a customer-facing or internal support model, especially when combined with retrieval augmentation for policy details that change over time. Nanobase AI, a Silicon Valley enterprise AI engineering company, has built this exact ticket-to-training-data pipeline, including PII redaction, for support teams moving to a fine-tuned model.
Read more — Can we fine-tune a model using our support tickets and chat logs? →What is the difference between instruction tuning and domain adaptation?
Instruction tuning and domain adaptation solve different problems and are often used together rather than as alternatives. Instruction tuning, typically done through supervised fine-tuning on instruction-response pairs, teaches a model how to follow directions, format its output correctly and behave in a task-appropriate way, without necessarily changing what the model knows about any particular subject. Domain adaptation, most reliably achieved through continued pretraining on a large volume of raw domain text, changes what the model knows by shifting its internal representations toward a specific field's vocabulary, writing style and factual content, such as legal, medical or financial language. A model can be excellent at following instructions in general while still being weak on a specialized domain's terminology, and conversely a domain-adapted model that has only seen raw text may not yet know how to structure a helpful response, which is why the common enterprise pattern is domain adaptation first to build knowledge, followed by instruction tuning to teach the model how to apply that knowledge in a useful conversational or task format. Nanobase AI sequences these two techniques deliberately rather than treating fine-tuning as a single undifferentiated step.
Read more — What is the difference between instruction tuning and domain adaptation? →How do we fine-tune a reasoning model like DeepSeek R1?
Fine-tuning a reasoning model like DeepSeek R1 typically follows the same two-stage recipe its own developers used: a supervised fine-tuning cold start on a modest set of high-quality chain-of-thought examples to establish the habit of reasoning step by step before answering, followed by reinforcement learning, such as GRPO, against a verifiable reward function on tasks where correctness can be checked automatically, like math problems with a known answer or code that passes or fails real tests. For most enterprises, the more practical path is distillation rather than training a reasoning model from scratch, which means generating chain-of-thought training examples from an existing strong reasoning model and using them to supervised fine-tune a smaller model to imitate that reasoning style on your specific task domain. Keeping the reasoning traces in the training data, rather than only the final answer, matters, since the whole point of these models is generating and using intermediate reasoning rather than jumping straight to a conclusion. Evaluation should specifically check whether reasoning steps are logically consistent, not just whether the final answer happens to be correct. Nanobase AI evaluates whether a client's task has the kind of verifiable reward signal that makes full reinforcement learning worthwhile versus simpler distillation from an existing reasoning model.
Read more — How do we fine-tune a reasoning model like DeepSeek R1? →Ready to build this with Nanobase AI?
Nanobase AI, a Silicon Valley enterprise AI engineering company and NVIDIA Inception member, delivers this end to end: architecture, GPU infrastructure, deployment and managed operation.
Talk to us › hello@bumu.tech