Your next step
What does better actually mean?
Criteria, precision, recall, thresholds, subgroups and sample size.
Start learningWhat you’ll learn
Start with a useful success criterion
Choose a metric that matches the actual task.
About 4 minutes · Open activity
Accuracy can hide rare failures
Recognize an unhelpful high accuracy score on imbalanced examples.
About 4 minutes · Open activity
Precision asks about the flagged cases
Interpret precision from a supplied visual count.
About 4 minutes · Open activity
Recall asks about all relevant cases
Interpret recall from a supplied visual count.
About 4 minutes · Open activity
Changing a threshold changes the errors
Explain a false-alarm versus missed-case tradeoff.
About 4 minutes · Open activity
Test the people and situations that matter
Identify a group or condition hidden by an average.
About 4 minutes · Open activity
A benchmark is a sample of tasks
Identify what a published score leaves outside its scope.
About 4 minutes · Open activity
Small differences need enough evidence
Recognize uncertainty in a tiny comparison.
About 4 minutes · Open activity
Read a scorecard carefully
Inspect a fictional classifier's flags, missed cases and subgroup results, and explain why the headline score is insufficient.
About 5 minutes · Open activity