Beyond Binary: Multiclass Classification

Every classifier you have met so far answers a yes-or-no question: sick or healthy, spam or not, churn or stay. It draws one boundary and picks a side. But most real problems have more than two answers.

Meena is a botanist. Each iris she pulls from a field tray must be labelled as one of THREE species: setosa, versicolor, or virginica. The flower in her hand has a petal 4.7 cm long and 1.4 cm wide. That is not a yes-or-no question; it is a one-of-three question, and the tools built for two classes do not obviously apply.

This lesson is how you get from a two-class classifier to a many-class one, and how the scorecard changes when there are more than two classes to keep track of.

By the end you will be able to:

• Turn a K-class problem into binary sub-problems with one-vs-rest and one-vs-one, and count how many models each needs
• Fit a natively multiclass model in R and read its per-class predictions
• Read a K-by-K confusion matrix and compute per-class precision, recall and F1
• Collapse those into one score with macro, micro and weighted averaging, and know which to trust when a class is rare
• Extend the ROC curve and AUC past two classes

Prerequisites: you can run R and read its output, and you have met a binary classifier that outputs a probability score, the confusion matrix, and precision, recall and ROC/AUC (the Reading a Classifier lesson).

The panel below is a single classifier separating two classes: the atom every idea in this lesson is built from. Drag the slider to watch it carve the boundary.

From two answers to many

A binary classifier's whole job is to split the world in two. It produces one score (a probability that the answer is "yes") and you compare that score to a threshold. One number, one boundary, two possible verdicts.

Meena's problem breaks that in a small but important way. She does not need a yes-or-no; she needs to pick exactly ONE label out of a fixed set. With \(K\) possible classes (here \(K = 3\)), a multiclass classifier takes an input and returns one of the \(K\) labels.

Two honest questions follow, and the rest of the lesson answers them in order:

• How do we train a classifier that can choose among three species, when the classifiers we know only say yes or no?
• How do we grade it, when "accuracy" is now spread across three classes that may not be equally easy or equally common?