Editing Probabilistic Machine Learning (section)

== <span style="color: #FFFFFF;">Applying</span> ==
'''Conformal prediction for guaranteed coverage:'''
<syntaxhighlight lang="python">
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

# Conformal prediction adds rigorous uncertainty quantification to any classifier
X, y = load_classification_dataset()
X_train, X_temp, y_train, y_temp = train_test_split(X, y, test_size=0.4)
X_cal, X_test, y_cal, y_test = train_test_split(X_temp, y_temp, test_size=0.5)

# Train base classifier
clf = RandomForestClassifier(n_estimators=100).fit(X_train, y_train)

# Calibration: compute nonconformity scores (1 - predicted prob of true class)
cal_probs = clf.predict_proba(X_cal)
cal_scores = 1 - cal_probs[np.arange(len(y_cal)), y_cal]  # Nonconformity scores

# Set coverage level
alpha = 0.1  # 90% coverage guarantee
threshold = np.quantile(cal_scores, (1 + 1/len(y_cal)) * (1 - alpha))

# Prediction sets for test examples
test_probs = clf.predict_proba(X_test)
def get_prediction_set(probs, threshold):
    return np.where(1 - probs <= threshold)[0]  # Include all classes with score ≤ threshold

prediction_sets = [get_prediction_set(p, threshold) for p in test_probs]
coverage = np.mean([y_test[i] in s for i, s in enumerate(prediction_sets)])
print(f"Coverage: {coverage:.2%} (target: {1-alpha:.0%})")  # Should be ≥ 90%
avg_set_size = np.mean([len(s) for s in prediction_sets])
print(f"Average prediction set size: {avg_set_size:.2f}")  # Smaller = more efficient
</syntaxhighlight>

; Probabilistic ML method selection
: '''Regression with uncertainty''' → Gaussian processes (small data), NGBoost, CARD
: '''Classification with calibration''' → Calibrated RF/XGBoost (Platt/isotonic); temperature scaling for DNN
: '''Guaranteed coverage''' → Conformal prediction (any model, any distribution)
: '''Generative modeling''' → VAE (smooth latent space), normalizing flows (exact likelihood), diffusion models
: '''Sequential inference''' → HMMs, Kalman filters, particle filters
</div>

<div style="background-color: #8B4500; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">