Editing Finetuning Llms (section)

== <span style="color: #FFFFFF;">Analyzing</span> ==
{| class="wikitable"
|+ Fine-tuning Method Comparison
! Method !! Params Updated !! GPU Memory !! Risk of Forgetting !! Quality
|-
| Full fine-tuning || 100% || Very high (multiple GPUs) || High || Highest
|-
| LoRA || 0.1–1% || Low (1 GPU possible) || Low || Near-full for most tasks
|-
| QLoRA || 0.1–1% (on 4-bit model) || Very low (fits on 24GB GPU) || Low || Slightly below LoRA
|-
| Prefix tuning || ~0.1% || Low || Very low || Moderate
|-
| Prompt tuning || ~0.01% || Very low || Very low || Lower than LoRA
|}

'''Failure modes:'''
* '''Overfitting on small datasets''' — With <500 examples, the model can memorize rather than generalize. Monitor validation loss; stop early.
* '''Format mismatch''' — Training on incorrectly formatted examples causes the model to generate malformed outputs or include spurious tokens.
* '''Instruction following collapse''' — Aggressive fine-tuning can make the model rigid, losing the flexibility to handle instructions it wasn't trained on.
* '''Reward hacking (RLHF)''' — The model learns to produce responses that score well according to the reward model without actually being more helpful — for example, becoming verbose without substance.
* '''Capability regression''' — Fine-tuning on a narrow task can degrade performance on other tasks. Evaluate on a broad benchmark before and after.
</div>

<div style="background-color: #483D8B; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">