Editing Natural Language Processing (section)

== <span style="color: #FFFFFF;">Evaluating</span> ==
Expert NLP practitioners evaluate systems holistically:

'''Intrinsic vs. extrinsic evaluation''': Intrinsic metrics (perplexity, BLEU) measure the model's internal properties. Extrinsic metrics measure performance on the downstream task. Always prioritize downstream task performance — a model with worse perplexity can outperform on the actual application.

'''Human evaluation''': For generation tasks, human raters evaluating fluency, coherence, factuality, and helpfulness remain the gold standard. A/B tests comparing model outputs are common in production.

'''Bias and fairness auditing''': Run the model on carefully constructed test sets designed to reveal disparate treatment. Tools like WinoBias (gender bias in coreference) are standard benchmarks.

'''Error analysis''': Rather than global metrics, inspect specific failure cases. Cluster errors by type (wrong entity span, wrong sentiment polarity, factual error). This guides targeted improvements more effectively than global metric optimization.

Expert practitioners distinguish between '''capability''' (what the model can do in optimal conditions) and '''reliability''' (how consistently it performs across diverse, realistic inputs). Production systems must optimize for reliability.
</div>

<div style="background-color: #2F4F4F; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">