Editing Transfer Learning (section)

== <span style="color: #FFFFFF;">Analyzing</span> ==
{| class="wikitable"
|+ Transfer Learning Strategy Comparison
! Strategy !! Labeled Data Needed !! Compute !! Risk of Overfitting !! Flexibility
|-
| Feature extraction || Very small (<500) || Very low || Very low || Low (head only)
|-
| Partial fine-tuning || Small (500–5k) || Low || Low || Medium
|-
| Full fine-tuning || Medium (5k+) || Medium || Medium || High
|-
| Train from scratch || Large (100k+) || Very high || Low (with enough data) || Maximum
|-
| Zero-shot transfer || None || None (inference only) || N/A || Moderate
|}

'''Failure modes and pitfalls:'''
* '''Negative transfer''' — When source and target domains are too dissimilar, pre-trained features actually hurt performance. Example: NLP models transfer poorly to genomic sequences; better to use domain-specific models.
* '''Catastrophic forgetting''' — Full fine-tuning on a small dataset can cause the model to "forget" general pre-trained knowledge. Mitigated by lower learning rates, fewer epochs, and progressive unfreezing.
* '''Data preprocessing mismatch''' — Pre-trained models expect a specific normalization. Using wrong mean/std values causes significant performance degradation even with correct weights.
* '''Label distribution shift''' — If the pre-training task had very different class balance than the target task, the model's feature priorities may be poorly aligned.
* '''Overconfident transfer''' — Assuming a pre-trained model from a similar domain will work without validation. Always run a baseline evaluation on target domain before assuming transferability.
</div>

<div style="background-color: #483D8B; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">