yardstick bal_accuracy() in R: Score Imbalanced Classifiers

The yardstick bal_accuracy() function in R returns the macro-average of per-class recall for a classification model, accepting a tibble of truth and prediction columns and producing a tidy one-row summary that stays honest when one class dominates the data.

bal_accuracy(df, truth, estimate)                       # basic two-class call
bal_accuracy(df, truth = obs, estimate = pred)          # named arguments
bal_accuracy(df, class, .pred_class)                    # default parsnip output
df |> group_by(fold) |> bal_accuracy(class, .pred_class)   # by resample
bal_accuracy(df, class, .pred_class, na_rm = TRUE)      # drop missing rows
bal_accuracy_vec(truth_vec, pred_vec)                   # vector interface
bal_accuracy(df, class, .pred_class, estimator = "macro_weighted")  # weighted macro

What bal_accuracy() measures

bal_accuracy() averages how well a model recalls each class. For a two-class problem it equals the mean of sensitivity and specificity. For multiclass it equals the macro average of per-class recall. The result is a number between 0 and 1; 0.5 is chance for a binary classifier.

The function accepts a data frame with two factor columns and returns a tibble with .metric, .estimator, and .estimate. It ignores class proportions: a model that always predicts the majority class scores zero on the minority class, dragging the average down.

Reach for balanced accuracy when one class is rare, costs are roughly symmetric, and you want a single headline number. Pair it with accuracy() so you can see how much the imbalance is hiding.

bal_accuracy() syntax and arguments

The signature is identical to every other yardstick class metric. The same call shape works whether you score logistic regression, a random forest, or a neural net.

Both truth and estimate must be factors with identical levels. The estimator argument only matters for multiclass: "macro" is the default; "macro_weighted" weights each class by prevalence, which partly defeats the point of using balanced accuracy.

Examples: balanced accuracy for binary and multiclass

The examples below build a deliberately imbalanced two-class data set, then a multiclass one. First, create a binary frame where 90 percent of rows belong to one class.

Example 1 contrasts accuracy with bal_accuracy on the same imbalanced data. Accuracy looks healthy because the model agrees with the dominant class most of the time. Balanced accuracy tells the real story.

Accuracy is 0.825 but balanced accuracy is barely above chance. The 82 percent figure is inflated by correct rejections of the majority class, not useful predictions on the rare class.

Example 2 scores a multiclass setup. For three or more classes, bal_accuracy defaults to the macro estimator: average the per-class recall, no weighting.

The .estimator column reports macro. Switch to estimator = "macro_weighted" to count each class proportional to its frequency.

Example 3 groups scoring by resample fold. With cross-validation predictions in one tibble, group_by() plus bal_accuracy returns one score per fold.

Example 4 uses the vector interface for ad-hoc checks. bal_accuracy_vec() takes two factors directly and returns a numeric scalar instead of a tibble.

The vector form is the right choice inside map() calls or unit tests.

bal_accuracy() compared with related metrics

Balanced accuracy lands between plain accuracy and the full precision-recall family. The table below summarizes when to reach for another yardstick metric.

A practical rule: report balanced accuracy as the headline whenever any class is below 30 percent of the data, and add roc_auc() if you have probability predictions.

Common pitfalls

Three small mistakes account for most bal_accuracy() errors. Each one has a clear fix.

The first is passing character vectors instead of factors. yardstick requires factors so it can validate that both columns share the same levels. The fix is one factor() call:

The second pitfall is forgetting the difference between "macro" and "macro_weighted". The default macro estimator weights every class equally, which is exactly what you want for imbalance. Switching to "macro_weighted" weights each class by its prevalence and re-introduces the very bias you were trying to escape.

The third pitfall is using event_level = "second" without noticing it flips the positive class. Sensitivity and specificity swap, the average stays numerically the same, but downstream precision() and recall() calls report a different class as positive. Set event_level once at the top of the workflow.

Try it yourself

Related yardstick metrics

bal_accuracy() is one entry in the yardstick class-metric family. Reach for these neighbors when balanced accuracy alone is not enough:

• accuracy() for the unbalanced baseline you compare against
• sens() and spec() for the two components of the binary case
• j_index() for Youden's J, the binary special case
• f_meas() for a precision-recall blend
• mcc() for the Matthews correlation coefficient
• roc_auc() and pr_auc() for probability-based ranking metrics

For the full set, see the yardstick reference index.

FAQ

How is balanced accuracy calculated for two classes?

For binary classification, bal_accuracy is the arithmetic mean of sensitivity and specificity. yardstick reads them from the confusion matrix and averages. The score is 1 when both rates are 1, 0.5 for a majority-only classifier, and 0 when the model is wrong on both classes.

When should I use bal_accuracy() instead of accuracy()?

Use balanced accuracy whenever class proportions are skewed and you care about the rare class. A 95 percent accuracy on a 95 / 5 split says little about the minority class, while balanced accuracy drops sharply if the model ignores it.

Does bal_accuracy() work for multiclass classification?

Yes. yardstick switches the .estimator from binary to macro based on the number of factor levels. The macro estimator averages per-class recall without weighting. Set estimator = "macro_weighted" if you want a prevalence-weighted average, though that partly undoes the rebalancing.

Can I compute balanced accuracy from a parsnip workflow?

Pipe augment() output into bal_accuracy. With augment(fit, new_data = test), the result already contains truth and .pred_class columns, so the call becomes augment(fit, test) |> bal_accuracy(class, .pred_class). Replace class with your outcome column name.

How does bal_accuracy compare with Cohen's kappa?

Balanced accuracy averages per-class recall; kap() compares observed agreement to chance agreement from marginal frequencies. Kappa can go negative when the model is worse than chance, while balanced accuracy is bounded at 0. Use kap for a chance-corrected agreement score.

Summary

bal_accuracy() is the right yardstick metric the moment one class dominates. Read it next to plain accuracy() so the gap quantifies the imbalance, and reach for roc_auc() or f_meas() when symmetric class treatment stops being the right framing. With group_by() it scales to any resampling scheme without reshaping.