Dear Mixomics Team,
Seetting a seed longer works for reproducibility. Please advise.
See code below.
choice.keepX is changing each time I run tune.splsda
###code
library(mixOmics) #‘6.31.4’
#PLSDA
set.seed(6419)
#splsda
list.keepX ← c(1:50)
#originally did 50
tune_splsda ← tune.splsda(X, Y, ncomp =2, validation = ‘Mfold’, folds = 3,
progressBar = TRUE, dist = ‘max.dist’, measure = “BER”,
test.keepX = list.keepX, nrepeat = 50)
tune_splsda$choice.keepX
Hi @jlabus,
Thanks for using mixOmics! You are currently using our latest version of mixOmics (6.31.4) which is under active development. For this new version of mixOmics, which will be released on Bioconductor in April, we have changed how we set the seed, instead of doing it outside the function you need to set seed as a argument in the tune.splsda function. We will share more details about the changes we have made to tune functions on our website soon, but in the meantime you can use the help files by running ?tune.splsda. We appreciate you using our latest version as you can help us in identifying any issues that may have arisen from the updates we have made! However if you would prefer to use the stable Bioconductor version of mixOmics (for example to keep your code consistent if you used that version before), you are welcome to do that too. I have written code below detailing how to install either the Github or Bioconductor versions of mixOmics and how to reproducibly run tune.splsda for both versions.
## set up example data
data(breast.tumors)
X = breast.tumors$gene.exp
Y = as.factor(breast.tumors$sample$treatment)
## mixOmics version 6.31.4 - development version on mixOmics
devtools::install_github("mixOmicsTeam/mixOmics", ref = "6.31.4", force = TRUE)
library(mixOmics)
# run tuning on test data twice with same seed set inside the function
tune = tune.splsda(X, Y, ncomp = 1, nrepeat = 5, folds = 10,
test.keepX = c(5, 10, 15),
seed = 20) # set for reproducibility of example only
tune$choice.keepX # 10
tune = tune.splsda(X, Y, ncomp = 1, nrepeat = 5, folds = 10,
test.keepX = c(5, 10, 15),
seed = 20) # set for reproducibility of example only
tune$choice.keepX # 10
## mixOmics version 6.30.0 - stable version on Bioconductor
BiocManager::install("mixOmics", force = TRUE)
library(mixOmics)
# run tuning on test data twice with same seed set outside function - creates reproducible result
set.seed(10)
tune = tune.splsda(X, Y, ncomp = 1, nrepeat = 5, folds = 10,
test.keepX = c(5, 10, 15))
tune$choice.keepX # 10
set.seed(10)
tune = tune.splsda(X, Y, ncomp = 1, nrepeat = 5, folds = 10,
test.keepX = c(5, 10, 15))
tune$choice.keepX # 10
Hope that helps!
Eva
Hi @evahamrud, thanks for addressing the question raised by @jlabus . I’ve come across a similar issue when using version 6.30.0. I’m only trying to run the following on my data using R v4.4.1:
set.seed(0)
result.plsda <- mixOmics::plsda(X, Y, ncomp = 2) # run the method with desired comps only
Yet, the results change every new day I rerun the code. That is, the plotting of the same samples with the same features are separated by a y=x line one day and a y=-x line the next with some data points moving around in the two clusters. Though, when run on the same day, the results do not change. I’m not sure why. I’ve also tried running v6.31.4 without any differences from v6.30.0 on the same day.
I will try to run the following and plot my results from today, then re-run the code tomorrow to show you the difference. If it helps any, running mixOmics’ pca does not seems to provide changing results, just the plsda function.
## set up example data
data(breast.tumors)
X = breast.tumors$gene.exp
Y = as.factor(breast.tumors$sample$treatment)
set.seed(0)
result.plsda <- mixOmics::plsda(X, Y, ncomp = 2) # run the method with desired comps only
mixOmics::plotIndiv(result.plsda, legend = TRUE, comp = c(1,2), title = 'Day 1, PLS_DA', )
plotIndiv(pca(X, ncomp = 2, scale = TRUE), group = Y, ind.names = FALSE,
legend = TRUE, title="Day 1, PCA")
Image:
Apologies, seems like there isn’t a day-to-day difference when using v6.31.4 or v6.30.0 for me anymore! It must have been something in my R session history changing things up. Since the plots on my own data haven’t been changing using v6.31.4 and setting the seed outside of the plotting function, I’ll just stick with that. Thanks!
