yardstick rmse() in R: Score Regression Models

The yardstick rmse() function in R returns the root mean squared error of a regression model, accepting a tibble with truth and estimate columns and producing a tidy one-row summary that reads in the same units as the outcome variable.

rmse(df, truth, estimate)                          # basic call
rmse(df, truth = obs, estimate = pred)             # named arguments
rmse(df, solubility, prediction)                   # default tidymodels output
df |> group_by(fold) |> rmse(obs, pred)            # by resample
rmse(df, obs, pred, na_rm = TRUE)                  # drop missing rows
rmse_vec(truth_vec, pred_vec)                      # vector interface
rmse(df, obs, pred, case_weights = w)              # weighted RMSE

What rmse() measures

rmse() squares each residual, averages them, then takes the square root. You pass a data frame with the observed numeric outcome and the predicted values, and the function returns a one-row tibble with three columns: .metric, .estimator, and .estimate. The estimate is a non-negative number on the same scale as the outcome, so an RMSE of 0.7 on a solubility score means predictions are off by roughly 0.7 units on average.

Because squaring punishes large misses, RMSE is more outlier-sensitive than mean absolute error. That sensitivity is a feature when one big miss matters more than many small ones, and a bug when noisy outliers dominate the score. Pair RMSE with mae() and rsq() whenever you compare regression models.

rmse() syntax and arguments

The signature matches every other yardstick numeric metric. Once you know the shape, the same call works for mae(), rsq(), mape(), and the rest of the regression family.

Truth and estimate must both be numeric; factor, character, or logical inputs raise an error. If you fitted a classification model, reach for accuracy() or roc_auc() instead.

Score regression models: four examples

The examples below use yardstick's built-in solubility_test data, which ships a real regression prediction set. First, load the package and inspect the data.

Example 1 calls rmse() with positional arguments. The function picks up truth and estimate by position and returns the tidy summary.

The .estimator column reports standard because RMSE has no binary or multiclass variant. The estimate of 0.722 is in log-solubility units.

Example 2 uses named arguments on a fresh prediction frame. Named arguments are clearer when truth and estimate sit next to other features in a larger tibble.

Example 3 groups scoring by resample fold. When predictions from cross-validation live in one tibble, group_by() plus rmse returns one score per fold, giving instant per-resample diagnostics.

Example 4 uses the vector interface for quick checks. Inside map() calls or unit tests, rmse_vec() returns a plain scalar instead of a one-row tibble.

Use the vector form when you need a scalar for thresholds or unit tests; otherwise stay with the data-frame form so you can bind, group, or plot scores.

rmse() compared with related metrics

RMSE is the most popular regression metric, but rarely the only one you should report. Pick a partner from the table when RMSE alone hides the story.

A safe default is RMSE as the headline number, MAE for outlier sensitivity, and R-squared for unit-free comparison across models with the same target.

Common pitfalls

Three small mistakes account for most rmse() failures. Each one has a one-line fix.

The first is passing a factor column. yardstick refuses factor inputs because a factor implies classification, not regression. The fix is to cast to numeric before scoring:

The second pitfall is comparing RMSE across different target transformations. An RMSE of 0.2 on log-price and 50 on raw-price are not on the same scale. Back-transform predictions before scoring, or stick to one target representation.

The third pitfall is missing values. With na_rm = TRUE (the default), yardstick silently drops NA rows, which changes the denominator. Pre-filter the prediction frame when comparing models that handle missingness differently.

Try it yourself

Related yardstick metrics

rmse() is one entry in the yardstick numeric-metric family. Reach for these neighbors when RMSE alone is not enough:

• mae() for an outlier-robust error in the same units
• mape() for percentage error reporting to stakeholders
• rsq() for a unit-free 0-to-1 goodness-of-fit
• rmsle() for skewed targets where multiplicative error matters
• ccc() for concordance correlation, useful in clinical and reliability work
• metrics() to compute several regression scores in a single call

For the full set, see the yardstick reference index.

FAQ

What is a good RMSE value?

There is no universal threshold. RMSE is in the units of your outcome, so a "good" value depends on the spread of that outcome. Compare RMSE against the standard deviation of the truth column: if RMSE is much smaller, the model is adding signal; if it is close, the model is barely better than predicting the mean. Always interpret alongside rsq() and a residual plot.

How is rmse() different from sqrt(mean((y - yhat)^2))?

They return the same number. yardstick wraps the formula in a function that validates inputs, handles missing values, supports case_weights, and returns a tidy tibble that integrates with metrics() and group_by(). Use rmse() everywhere for consistency with the tidymodels workflow.

Can I weight rows when computing RMSE?

Yes. Pass a case_weights column to rmse(). Common uses are survey weights, sample weights from parsnip::fit(), or importance weights. The result is a weighted root mean squared error on the outcome scale. Weights must be non-negative.

Why does rmse() return a tibble instead of a number?

The tidy return shape is the yardstick convention. Every metric returns the same three columns: .metric, .estimator, .estimate. That uniformity lets you bind_rows() calls or pipe into group_by() with no reshape step. Call rmse_vec() when you only need the scalar.

Should I prefer RMSE or MAE for tuning?

If you care most about avoiding large individual errors, tune on RMSE. If your business cost is roughly linear in error size, tune on MAE. The difference matters when the error distribution is heavy-tailed: RMSE favours models that flatten the tail; MAE favours models that minimise typical error.

Summary

rmse() is the default regression scorecard in yardstick. Use it for the headline number, lean on mae() and rsq() for partners, and switch to rmse_vec() only when you need a scalar. Combined with group_by() it scales to any resampling scheme without reshaping, and combined with bind_rows(metrics(...)) it produces a clean multi-metric report ready to publish.