Skip to content

Instantly share code, notes, and snippets.

@PSanni
PSanni / noul_cal.md
Created November 3, 2026 23:55
Sarvams prism vs Jev (Diagnosing Noul calibration with schema classification)

Method

Expected calibration error. Ten equal-width bins over P(yes):

$$\text{ECE}=\sum_{b=1}^{10}\frac{n_b}{N},\bigl|,\bar p_b-\bar y_b,\bigr|$$

where $\bar p_b$ is the mean stated probability in bin $b$ and $\bar y_b$ is the share actually gold.

Corrections. Applied on the log-odds, fitted by logistic regression: