[Bug]: ordering of selected in choices_selected is not respected

[crystalluckett-sanofi]

What happened?

Reprex:

data <- teal_data()
data <- within(data, {
  ADSL <- tmc_ex_adsl
  ADSL$EOSDY[1] <- NA_integer_
})
join_keys(data) <- default_cdisc_join_keys[names(data)]

ADSL <- data[["ADSL"]]

app <- init(
  data = data,
  modules = modules(
    tm_t_summary(
      label = "Demographic Table",
      dataname = "ADSL",
      arm_var = choices_selected(c("ARM", "ARMCD"), "ARM"),
      add_total = TRUE,
      summarize_vars = choices_selected(
        variable_choices(ADSL),
        c("SEX", "REGION1", "STRATA1")
      ),
      useNA = "ifany"
    )
  )
)
if (interactive()) {
  shinyApp(app$ui, app$server)
}

Expected behavior:

• all variables in ADSL as choices in dropdown
• SEX, REGION1, and STRATA selected in that order

Actual behavior:
Variables are selected in the order they appear in the dataset

I believe it may be due to this line

teal.transform/R/choices_selected.R

Lines 174 to 177 in a535ca0

	choices <- vector_reorder(
	choices,
	c(choices_in_selected, setdiff(seq_along(choices), choices_in_selected))
	)

Perhaps this is not a bug and just your desired behavior, but it seems to me that the ordering of the selected argument should be respected in the UI upon launch. Currently, to get around this, I can specify the subset argument in variable_choices. However, if I wish to have all ADSL variables, specifying all of them in some preferred order seems like it should not be necessary.

sessionInfo()

R version 4.2.1 (2022-06-23)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: CentOS Linux 7 (Core)

Matrix products: default
BLAS/LAPACK: /fsx-perm/SERVICES/WISE/easybuild/software/OpenBLAS/0.3.12-GCC-10.2.0/lib/libopenblas_skylakexp-r0.3.12.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=en_US.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=en_US.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices datasets  utils     methods  
[7] base     

other attached packages:
 [1] teal.modules.clinical_0.10.0 tern_0.9.7                  
 [3] rtables_0.6.12               magrittr_2.0.3              
 [5] formatters_0.5.11            teal.transform_0.6.0        
 [7] teal_0.16.0                  teal.slice_0.6.0            
 [9] teal.data_0.7.0              teal.code_0.6.1             
[11] shiny_1.10.0                

loaded via a namespace (and not attached):
  [1] nlme_3.1-164          geeasy_0.1.2         
  [3] teal.logger_0.3.2     fontawesome_0.5.3    
  [5] RColorBrewer_1.1-3    numDeriv_2016.8-1.1  
  [7] tools_4.2.1           TMB_1.9.17           
  [9] backports_1.5.0       bslib_0.9.0          
 [11] R6_2.6.1              DT_0.33              
 [13] mgcv_1.9-1            MESS_0.5.12          
 [15] withr_3.0.2           tidyselect_1.2.1     
 [17] shinyvalidate_0.1.3   emmeans_1.11.1       
 [19] compiler_4.2.1        cli_3.6.5            
 [21] formatR_1.14          shinyjs_2.1.0        
 [23] sandwich_3.1-1        sass_0.4.9           
 [25] mosaicCore_0.9.4.0    scales_1.4.0         
 [27] checkmate_2.3.2       mvtnorm_1.3-3        
 [29] ggridges_0.5.6        stringr_1.5.1        
 [31] digest_0.6.37         ggformula_0.12.0     
 [33] minqa_1.2.8           rmarkdown_2.29       
 [35] pkgconfig_2.0.3       htmltools_0.5.8.1    
 [37] lme4_1.1-37           teal.widgets_0.4.3   
 [39] geeM_0.10.1           labelled_2.14.0      
 [41] fastmap_1.2.0         htmlwidgets_1.6.4    
 [43] rlang_1.1.6           rstudioapi_0.17.1    
 [45] jquerylib_0.1.4       farver_2.1.2         
 [47] generics_0.1.4        zoo_1.8-13           
 [49] jsonlite_1.9.1        dplyr_1.1.4          
 [51] Matrix_1.6-0          Rcpp_1.0.14          
 [53] shinycssloaders_1.1.0 clipr_0.8.0          
 [55] geepack_1.3.12        logger_0.4.0         
 [57] lifecycle_1.0.4       stringi_1.8.7        
 [59] multcomp_1.4-28       yaml_2.3.10          
 [61] MASS_7.3-60.0.1       grid_4.2.1           
 [63] tern.gee_0.1.5        promises_1.3.2       
 [65] forcats_1.0.0         lattice_0.22-6       
 [67] haven_2.5.4           splines_4.2.1        
 [69] nestcolor_0.1.3       hms_1.1.3            
 [71] knitr_1.50            pillar_1.10.2        
 [73] boot_1.3-30           estimability_1.5.1   
 [75] codetools_0.2-20      glue_1.8.0           
 [77] evaluate_1.0.3        renv_1.1.4           
 [79] vctrs_0.6.5           nloptr_2.2.1         
 [81] httpuv_1.6.15         Rdpack_2.6.4         
 [83] gtable_0.3.6          purrr_1.0.4          
 [85] tidyr_1.3.1           cachem_1.1.0         
 [87] ggplot2_3.5.2         xfun_0.51            
 [89] mime_0.13             rbibutils_2.3        
 [91] xtable_1.8-4          broom_1.0.8          
 [93] teal.reporter_0.4.0   later_1.4.1          
 [95] reformulas_0.4.1      rsconnect_1.3.4      
 [97] survival_3.6-4        tibble_3.3.0         
 [99] glmmTMB_1.1.10        memoise_2.0.1        
[101] shinyWidgets_0.9.0    TH.data_1.1-3

Relevant log output

Code of Conduct

• I agree to follow this project's Code of Conduct.

Contribution Guidelines

• I agree to follow this project's Contribution Guidelines.

Security Policy

• I agree to follow this project's Security Policy.

[llrs-roche]

Hi @crystalluckett-sanofi, sorry for not getting back to you sooner.
How we handle the column specification and extraction is being updated from scratch (See prove of concept on #256) . It is not always easy to know the order of the output when selecting variables, specially if input tables are merged or there are multiple selections from the same inputs (Changes on teal.transform::choices_selected will affect all/most teal packages). We will keep this in mind and not change the user preference (if possible).

However, reading the code of vector_reorder I think this function was written to keep the order of the vector as specified on choices parameter (while keeping attributes). It might be that it doesn't work well or that any other process between that and the output changes the order (so it might be a module specific issue).

[crystalluckett-sanofi]

Hi @llrs-roche, similar apologies for taking awhile to get back to you.
Is there any update on this?
Would this be a possible solution that you would find acceptable?

Using the rADSL dataset, the order of some columns is SEX then REGION1 then STRATA1, but we may wish to select them in the order SEX, STRATA1, then REGION1.

Applying the vector_reorder() function, the result is that items are selected in the order of the dataset, as seen in choices

#dataset
ADSL <- teal.data::rADSL
#column order has SEX > REGION1 > STRATA1

# teal.transform
choices <- colnames(ADSL)
selected <- c("SEX", "STRATA1", "REGION1")

choices_in_selected <- which(choices %in% selected)
choices <- teal.transform:::vector_reorder(
  choices,
  c(choices_in_selected, setdiff(seq_along(choices), choices_in_selected))
)

Alternatively, the following code produces a version of choices that has the selected variables in the order they were selected.

choices <- colnames(ADSL)
selected <- c("SEX", "STRATA1", "REGION1")

choices_in_selected <- intersect(selected, choices)
choices <- union(choices_in_selected, choices)

[llrs-roche]

Hi! Sorry, I was busy preparing for some releases.

Double checking the issue, it seems that even with keep_order = TRUE on choices_selected():

summarize_vars = choices_selected(
        variable_choices(ADSL),
        c("SEX", "REGION1", "STRATA1"),
		keep_order = TRUE
      ),

The order is not kept and this should be fixed.

A PR would be appreciated as the team is busy, as long as the PR is minimal (along the lines you suggest here) I'm sure we'll accepted.

Note however, that to mark the problem as solved first we'll need to make sure the issue is on choices_selected() instead of on the module tm_t_summary().