Editing Finetuning Llms (section)

== <span style="color: #FFFFFF;">Creating</span> ==
Designing a full fine-tuning pipeline:

'''1. Dataset curation (most important step)'''
<syntaxhighlight lang="text">
Source data collection (domain documents, logs, demonstrations)
    ↓
Quality filtering (deduplication, length filtering, toxic content removal)
    ↓
Formatting (convert to chat template, add system prompt)
    ↓
Review sample (manually inspect 100+ examples)
    ↓
Train/validation split (90/10 or 95/5)
</syntaxhighlight>

'''2. Training configuration decision tree'''
* <1k examples and 1 GPU → QLoRA with early stopping
* 1k–100k examples and 2–8 GPUs → LoRA with gradient checkpointing
* >100k examples and production budget → Full fine-tune with DDP/FSDP

'''3. Iterative refinement loop'''
<syntaxhighlight lang="text">
v1: SFT on demonstrations
    ↓ evaluate → identify failure cases
v2: Add failure case examples to dataset, retrain
    ↓ evaluate → identify preference gaps
v3: Collect human preference data → train reward model → PPO/DPO fine-tune
</syntaxhighlight>

'''4. Serving the fine-tuned model'''
* Merge LoRA adapters into base model: <code>model.merge''and''unload()</code>
* Export to GGUF format for llama.cpp (local/edge deployment)
* Push to HuggingFace Hub or deploy with vLLM for API serving

[[Category:Artificial Intelligence]]
[[Category:Large Language Models]]
[[Category:Machine Learning]]
</div>