Nested cross validation

Hi, we are testing DIABLO for our analysis and want to try a nested cross validation approach to build a discriminatory signature. Our dataset is small with 60 samples, would this dataset size be appropriate for a nested model?

Also, how do we select on the best hyperparameters to select the final model based on prediction of each outer loop validation set?

would this dataset size be appropriate for a nested model?

I would think that 60 samples is sufficient for nested CV, but it depends on the number of folds you are planning to use in the inner and outer CV loops.

how do we select on the best hyperparameters to select the final model based on prediction of each outer loop validation set?

In a similar vain as to how the tune.*() functions work within mixOmics: at each iteration record the predictions and how they compare to the ground truths (giving you an error rate, ER). Save the ER at each iteration. Then for each permutation of hyperparameters, you will have a distribution of ERs. You can simply select that with the lowest mean/median ER, or you could use T-tests to determine the optimal hyperparameter values