diff --git a/content/blog/tidymodels-april-2026/index.qmd b/content/blog/tidymodels-april-2026/index.qmd index 35e93a991..4393b378c 100644 --- a/content/blog/tidymodels-april-2026/index.qmd +++ b/content/blog/tidymodels-april-2026/index.qmd @@ -32,12 +32,11 @@ We've released a sequence of tidymodels packages over the last few weeks: dials ```{r} #| label: versions #| eval: false - # tidymodels installs all of the new versions pak::pak("tidymodels") ``` -Let's first talk about the two biggest updates enabled by this group of releases then we'll cover some of the other changes for each package. +Let's first talk about the two biggest updates enabled by this group of releases, then we'll cover some of the other changes for each package. ## Ordered Outcomes @@ -50,7 +49,7 @@ The [ordered package by Cory Brunson](https://github.com/corybrunson/ordered) is - `decision_tree()`: `"rpartScore"` - `rand_forest()`: `"ordinalForest"` -These models can be fit, tuned, and evaluated with tidymodels. For the evaluation, we've added a specific performance metric for ordered categories: the [ranked probability score](https://aml4td.org/chapters/cls-metrics.html#sec-ordered-categories) (RPS). The function `ranked_prob_score()` is in the new yardstick release and requires an ordered factor for the outcome. +These models can be fitted, tuned, and evaluated with tidymodels. For the evaluation, we've added a specific performance metric for ordered categories: the [ranked probability score](https://aml4td.org/chapters/cls-metrics.html#sec-ordered-categories) (RPS). The function `ranked_prob_score()` is in the new yardstick release and requires an ordered factor for the outcome. ## Quantile Regression @@ -59,7 +58,6 @@ We [previously reported](https://tidyverse.org/blog/2025/02/tidymodels-2025-q1/# ```{r} #| label: startup #| include: false - library(tidymodels) library(future) @@ -76,22 +74,21 @@ Here's a simple one-dimensional example using the Ames data; we'll predict the s #| fig-align: center #| out-width: 90% #| message: false - library(tidymodels) # We'll also need the qrnn package for the neural network engine set.seed(1215) -ames_split <- - ames |> - select(Latitude, Sale_Price) |> +ames_split <- + ames |> + select(Latitude, Sale_Price) |> initial_split(strata = Sale_Price) ames_train <- training(ames_split) ames_test <- testing(ames_split) ames_rs <- vfold_cv(ames_train, strata = Sale_Price) -ames_train |> - ggplot(aes(Latitude, Sale_Price)) + - geom_point(alpha = 1 / 5) + +ames_train |> + ggplot(aes(Latitude, Sale_Price)) + + geom_point(alpha = 1 / 5) + geom_smooth(se = FALSE) + labs(x = "Latitude", y = "Sale Price (USD)") ``` @@ -102,22 +99,21 @@ There are a few engines for quantile regression, and we'll use a neural network ```{r} #| label: quantile-spec - # Pre-defined quantiles of interest qnt_lvls <- c(0.05, 0.25, 0.5, 0.75, 0.95) -nnet_spec <- - mlp(hidden_units = tune(), penalty = tune(), epochs = 10) |> +nnet_spec <- + mlp(hidden_units = tune(), penalty = tune(), epochs = 10) |> # Set the quantile levels with the mode: - set_mode("quantile regression", quantile_levels = qnt_lvls) |> - # A new engine for quantile regression with neural networks via the - # qrnn package. We'll add an engine argument to specify the + set_mode("quantile regression", quantile_levels = qnt_lvls) |> + # A new engine for quantile regression with neural networks via the + # qrnn package. We'll add an engine argument to specify the # optimization method for training the model: set_engine("qrnn", method = "adam") -# Scale the single predictor to help the model initialize its -# parameters. -nnet_rec <- recipe(Sale_Price ~ ., data = ames_train) |> +# Scale the single predictor to help the model initialize its +# parameters. +nnet_rec <- recipe(Sale_Price ~ ., data = ames_train) |> step_normalize(all_predictors()) nnet_wflow <- workflow(nnet_rec, nnet_spec) @@ -130,8 +126,8 @@ We'll use a small grid: ```{r} #| label: quantile-tune set.seed(971) -nnet_res <- - nnet_wflow |> +nnet_res <- + nnet_wflow |> tune_grid( resamples = ames_rs, grid = 25, @@ -146,12 +142,11 @@ We can get the performance metric and visualize which tuning parameter combinati #| fig-width: 5 #| fig-height: 5 #| fig-align: center - nnet_mtr <- collect_metrics(nnet_res) -nnet_mtr |> - ggplot(aes(penalty, hidden_units, size = mean)) + - geom_point() + +nnet_mtr |> + ggplot(aes(penalty, hidden_units, size = mean)) + + geom_point() + scale_x_log10() + coord_fixed(ratio = 1) + labs(x = "Penalty", y = "# Hidden Units", size = "WIS") @@ -178,9 +173,9 @@ worst_model <- set.seed(8281) mid_model <- - nnet_mtr |> + nnet_mtr |> # Since we have an odd number of grid points: - filter(mean == median(mean)) |> + filter(mean == median(mean)) |> select(hidden_units, penalty) |> finalize_workflow(nnet_wflow, parameters = _) |> fit(ames_train) @@ -194,30 +189,30 @@ Now let's plot the results. We'll color the predicted quantiles: black indicates #| fig-height: 3 #| fig-align: center bind_rows( - best_model |> augment(ames_test) |> mutate(Model = "Best Results"), - mid_model |> augment(ames_test) |> mutate(Model = "Meh Results"), - worst_model |> augment(ames_test) |> mutate(Model = "Worst Results") - ) |> + best_model |> augment(ames_test) |> mutate(Model = "Best Results"), + mid_model |> augment(ames_test) |> mutate(Model = "Meh Results"), + worst_model |> augment(ames_test) |> mutate(Model = "Worst Results") +) |> mutate( .pred_quantile = map(.pred_quantile, ~ as_tibble(.x)) - ) |> - unnest(.pred_quantile) |> - arrange(Latitude) |> - ggplot(aes(Latitude)) + - geom_point(aes(y = Sale_Price), alpha = 1 / 30, cex = 3 / 4) + + ) |> + unnest(.pred_quantile) |> + arrange(Latitude) |> + ggplot(aes(Latitude)) + + geom_point(aes(y = Sale_Price), alpha = 1 / 30, cex = 3 / 4) + geom_path( aes( - y = .pred_quantile, - group = .quantile_levels, + y = .pred_quantile, + group = .quantile_levels, col = factor(.quantile_levels) - ), + ), show.legend = FALSE, linewidth = 1 ) + scale_color_manual( values = c("#8785B2FF", "#D95F30FF", "black", "#D95F30FF", "#8785B2FF") - ) + - facet_wrap(~ Model) + ) + + facet_wrap(~Model) ``` These plots show that configurations with very large score values have poor fits (linear in this case). The "meh" model is nonlinear but not responsive enough to the datas' ups and downs. The best model, with more hidden units and a low penalty, appears to be flexible enough to model the data well. @@ -228,17 +223,20 @@ Now we'll describe various other improvements in the recently released versions. ## dials +The latest dials release contains several new parameters for new-ish models in parsnip: For the `ordinal_reg()` models, dials now contains `ordinal_link()` and `odds_link()`. For the `tab_pfn()`, dials contains `num_estimators()`, `softmax_temperature()`, `balance_probabilities()`, `average_before_softmax()`, and `training_set_limit()`. + +The other user-facing changes were related to input checking and related error messages. The most prominent example is that `parameters()` and the `grid_*()` functions now give more information in the error message when non-parameter objects are passed in: which inputs aren't a parameter object and what they are instead. [@corybrunson](https://github.com/corybrunson), [@daltonkw](https://github.com/daltonkw), [@hfrick](https://github.com/hfrick), [@jeroenjanssens](https://github.com/jeroenjanssens), [@topepo](https://github.com/topepo), and [@vmikk](https://github.com/vmikk) contributed to the package since the last release. ## yardstick -We are thankful to the developers who ocntributed to this version: [@abichat](https://github.com/abichat), [@astamm](https://github.com/astamm), [@corybrunson](https://github.com/corybrunson), [@DarioS](https://github.com/DarioS), [@EmilHvitfeldt](https://github.com/EmilHvitfeldt), [@FvD](https://github.com/FvD), [@hfrick](https://github.com/hfrick), [@JavOrraca](https://github.com/JavOrraca), [@jeroenjanssens](https://github.com/jeroenjanssens), [@jkylearmstrong-temple](https://github.com/jkylearmstrong-temple), [@mle2718](https://github.com/mle2718), [@nathant181](https://github.com/nathant181), [@SimonDedman](https://github.com/SimonDedman), [@topepo](https://github.com/topepo), and [@tripartio](https://github.com/tripartio) +We are thankful to the developers who contributed to this version: [@abichat](https://github.com/abichat), [@astamm](https://github.com/astamm), [@corybrunson](https://github.com/corybrunson), [@DarioS](https://github.com/DarioS), [@EmilHvitfeldt](https://github.com/EmilHvitfeldt), [@FvD](https://github.com/FvD), [@hfrick](https://github.com/hfrick), [@JavOrraca](https://github.com/JavOrraca), [@jeroenjanssens](https://github.com/jeroenjanssens), [@jkylearmstrong-temple](https://github.com/jkylearmstrong-temple), [@mle2718](https://github.com/mle2718), [@nathant181](https://github.com/nathant181), [@SimonDedman](https://github.com/SimonDedman), [@topepo](https://github.com/topepo), and [@tripartio](https://github.com/tripartio) ## parsnip -Version 1.5.0 of parsnip had a variety of changes. Besides the additions for the two new model types shownn above: +Version 1.5.0 of parsnip had a variety of changes. Besides the additions for the two new model types shown above: - We enabled case weight usage for the `"nnet"` engines of `mlp()` and `bag_mlp()` as well as for the `"dbarts"` engine of `bart()`. @@ -254,7 +252,9 @@ Thanks to those who contributed to parsnip since the last release: [@Cere ## tune -Hannah - do you want to write about the optimizations here? +The core functionality of tune is to do all the model fitting (including pre- and postprocessing) and performance evaluation across various resamples and tuning parameter combinations. For grid search, we could take the full parameter grid, splice one parameter combination into the workflow at a time, and run with it. That can be pretty inefficient though. So what actually happens in tune are a few optimizations in how we do all that fitting and evaluating: +For preprocessing, we do it once for a resample (per preprocessing parameter combination) and then evaluate all model candidates on it. This lets us avoid unnecessarily repeating the same preprocessing multiple times. +For model fitting, we make use of what Max calls ["the submodel trick"](https://parsnip.tidymodels.org/articles/Submodels.html): For certain models, like a boosted tree, you can use _a submodel_ to make predictions without having to refit the model. A boosted tree ensemble fitted with 20 trees can be used to make predictions for any number of trees up to the 20 used for fitting. That allows us to evaluate different tuning parameter candidates for, here, the number of trees, without having to refit the model. When we added postprocessing, we temporarily disabled this (to ensure we got the integration right) - now we've brought it back. We make use of this speedup for both the main model as well as the calibration model. One big update is that the Gaussian process model package was changed from GPfit to GauPro because the former is no longer actively maintained. There are some differences: