tune tune_bayes() in R: Bayesian Hyperparameter Search

The tune tune_bayes() function in R runs iterative Bayesian optimization over a tidymodels workflow, fitting a Gaussian process surrogate on past results to propose smarter candidates each round instead of scoring a fixed grid.

tune_bayes(wf, resamples = folds)                            # default 5-point init + 10 iter
tune_bayes(wf, resamples = folds, iter = 25)                 # longer search
tune_bayes(wf, resamples = folds, initial = 15)              # bigger initial design
tune_bayes(wf, resamples = folds, initial = init_res)        # reuse a prior tune_grid()
tune_bayes(wf, resamples = folds, metrics = mset)            # custom metric set
tune_bayes(wf, resamples = folds, objective = exp_improve()) # expected improvement
tune_bayes(wf, resamples = folds, param_info = params)       # custom ranges
tune_bayes(wf, resamples = folds, control = ctrl_bayes)      # verbose, no_improve, parallel

What tune_bayes() does in one sentence

tune_bayes() learns where the metric is best, then samples there. You give it a workflow whose model spec contains tune() placeholders, a resample object, and an iteration budget. The function first scores a small space-filling initial design, fits a Gaussian process to the metric values, then repeatedly maximizes an acquisition function (expected improvement by default) to pick the next candidate. Each iteration adds one row to the surrogate and refits it, so the search concentrates on promising regions instead of treating every grid point equally.

The return value is a tune_results object identical in shape to tune_grid(), which means collect_metrics(), show_best(), select_best(), and finalize_workflow() work without changes. The difference is fits per run: a 15-point grid scored on 10 folds is 150 fits, while a tune_bayes() call with 5 initial points and 10 iterations is 150 fits scattered intelligently across the parameter space.

Set up a tunable workflow

You need three pieces before calling tune_bayes(). A model spec with tune() placeholders, a recipe or formula, and a resample object that splits the training data. Continuous parameters with broad ranges benefit most from Bayesian search.

The example below tunes a random forest via the ranger engine. Three knobs vary at once: mtry, min_n, and trees. With nine possible values of mtry, ten of min_n, and a continuous trees range, an exhaustive grid quickly explodes; Bayesian search stays tractable.

tune_bayes() syntax and arguments

The signature is short, but a few arguments do most of the work.

Examples by use case

Start with the simplest Bayesian call, then refine. Finalize the parameter set first so mtry has a concrete upper bound.

Once results are back, the helpers behave exactly as with grid search.

Reuse a prior tune_grid() call as the initial design when you already have grid results, which avoids paying for a fresh space-filling sample.

Switch the acquisition function when the search keeps revisiting the same neighborhood. conf_bound() with a larger kappa pushes the next pick away from current best estimates.

tune_bayes() versus tune_grid() and tune_sim_anneal()

Pick by parameter type and fit cost, not by sophistication.

Bayesian search shines when fits are slow and ranges are wide. With a 50ms fit, tune_grid(grid = 50) beats tune_bayes(iter = 30) on wall clock; with a 30s fit, the calculus flips.

Common pitfalls

Three errors account for most failed Bayesian runs.

1. Unfinalized parameter set. Calling tune_bayes() without finalizing mtry or any other data-dependent range throws ! Some parameter values are unknown. Always pass param_info built with finalize(train).
2. Too few initial points. A surrogate fit on three candidates is meaningless; the search wanders. Use at least one initial point per tunable parameter, ideally more.
3. Confusing iterations with fits. iter = 25 means 25 single candidates added on top of initial. Total fits is (initial + iter) * folds, not iter * folds. Budget accordingly.

Try it yourself

Related tidymodels functions

Most tune_bayes() runs sit inside a short chain of helpers.

• tune_grid() to score a fixed candidate set or seed a Bayesian warm start.
• fit_resamples() to resample a workflow with no tunable arguments.
• last_fit() to refit the finalized workflow on the full train and score the test set.
• finalize_workflow() and select_best() to lock in the winning hyperparameters.
• extract_parameter_set_dials() and finalize() to discover and resolve parameter ranges.
• control_bayes() to toggle verbose iteration logs, early stopping, parallel backend, and prediction saving.

External reference: the official tune package documentation at tune.tidymodels.org.

FAQ

How many iterations should I set for tune_bayes()?

Budget the initial design first, then add iterations as patience allows. A good rule is five initial points per tunable parameter and fifteen to thirty iterations on top. If control_bayes(no_improve = 10) is set, the search exits early when the surrogate stops finding gains, so over-budgeting is mostly free. With one-second fits and ten folds, sixty total candidates take roughly ten minutes; scale linearly from there.

What is the difference between exp_improve() and conf_bound()?

exp_improve() weights candidates by expected gain over the current best metric. It exploits when the surrogate is confident and explores otherwise. conf_bound(kappa) adds kappa standard deviations to the surrogate mean, so larger kappa forces exploration. Default is exp_improve(); switch to conf_bound(kappa = 2) when the search collapses to a single neighborhood and you suspect a better region exists elsewhere.

Can I run tune_bayes() in parallel?

Yes, but iteration logic is sequential by nature. Register a backend before the call (doFuture plus plan(multisession, workers = 4)) and the function parallelizes folds within each iteration, not iterations themselves. Set control_bayes(parallel_over = "everything") only when the inner per-iteration grid is large; for typical single-candidate iterations, "resamples" is fastest.

Why does the initial design matter so much?

The surrogate's quality depends on how well the initial points cover the space. Too few points or a clustered design produce a misleading mean surface, and the acquisition function chases noise. Use at least one point per parameter, prefer space-filling designs over random ones, and reuse a previous tune_grid() result via initial = prior_res when you have one.

Does tune_bayes() always beat tune_grid()?

No. With one or two discrete parameters and cheap fits, an exhaustive grid is faster and reports more uniformly. Bayesian search wins when the parameter space is continuous, the dimension is at least three, fits cost seconds or more, and you accept that results are no longer a uniform map of the metric surface. Prototype with tune_grid(grid = 10), then escalate to tune_bayes() once the ranges feel right.