Chapter C41 · Interactive Course
Test reliability beyond a headline score
Fair conditions, contamination, robustness, calibration, judges.
Core Learning Concept
Audit the demo’s comparison, data independence, robustness, judging method, and safety probes. Identify what each check actually establishes and where evidence is missing. Use harmless prepared examples for boundary tests. The result should be a bounded assessment of this setup, not a promise about every possible future input.
What You’ll Learn in This Chapter
-
Keep evaluation conditions comparable
Identify an unfair model comparison.
A model-only comparison should hold relevant conditions steady, including inputs, tools, budget, and judging. A product comparison may intentionally compare complete setups instead. Both can be useful if labeled accurately. Avoid attributing a difference to model quality when several other factors changed at the same time.
-
A memorized exam is weak evidence
Detect benchmark contamination.
A test that overlaps training can overstate performance on unfamiliar tasks. Public availability creates a contamination question, not proof that every item was memorized. Inspect collection dates, deduplication, and development use, and add independent representative cases where possible. A newer test still needs quality and leakage checks.
-
Robustness means handling relevant changes
Choose a meaningful perturbation for a test.
Robustness is useful performance under relevant changes, such as natural paraphrases, noise, or different formatting. Keep the intended task and correct answer unchanged while varying the input. Compare behavior across those variations. A response can change wording harmlessly; the concern is whether required meaning or functionality fails.
-
Confidence needs calibration evidence
Distinguish an evaluated confidence estimate from confident wording.
Calibration compares predicted confidence with actual outcomes across many cases. If a model assigns 80% confidence, about 80% of those answers should be correct. Never trust confident phrasing without checking empirical accuracy.
-
An explanation may be unfaithful
Separate a useful answer explanation from a verified account of computation.
A generated explanation can be useful reasoning to inspect without being a faithful record of internal computation. Check its factual and logical claims independently. A plausible story about why an answer occurred does not prove that causal account. External traces and controlled tests answer different evidence questions.
-
AI judges also need evaluation
Identify a bias or criterion mismatch in an automated evaluator.
An AI judge uses a model to evaluate other outputs. It can introduce subtle biases, length preferences, or shared blind spots. Always validate automated judges against human gold standards and tricky edge cases.
-
Probe boundaries safely
Recognize the purpose of adversarial testing on fictional material.
Boundary testing probes defined limits with authorized, harmless examples. Vary wording or context to see whether the intended behavior holds, while controlling access and side effects. Record the setup and observed result. Passing a bounded probe is useful evidence for that case, not proof against all possible failures.
-
Audit a model demo
Find an unfair test condition, a weak evidence claim and an untested reliability boundary in a fictional product demonstration.