DIABLO: Error while shutting down parallel unable to terminate some child processes

Bugs
1

Hi,

I am using DIABLO to do a multi-omics analysis using multiple CPUs. I have 14 samples in total. Here is the error message.

$DEG
[1] 14 914

$DMR
[1] 14 677

$Metabolite
[1] 14 54

AA EA
8 6
DEG DMR Metabolite
DEG 0.0 0.5 0.1
DMR 0.5 0.0 0.1
Metabolite 0.1 0.1 0.0
Design matrix has changed to include Y; each block will be
linked to Y.
max.dist centroids.dist mahalanobis.dist
Overall.ER 1 1 1
Overall.BER 1 1 1

You have provided a sequence of keepX of length: 905 for block DEG and 668 for block DMR and 45 for block Metabolite.
This results in 27204300 models being fitted for each component and each nrepeat, this may take some time to run, be patient!
Error in checkForRemoteErrors(val) :
50 nodes produced errors; first error: unable to fork, possible reason: Cannot allocate memory
Calls: tune.block.splsda … clusterApply -> staticClusterApply -> checkForRemoteErrors
Execution halted
Error while shutting down parallel: unable to terminate some child processes

For the DEG dataset, there are 914 differential expression genes.
For the DMR dataset, there are 677 differential methylation regions.
For the Metabolite dataset, there are 54 differential metabolites.

Screen Shot 2021-01-11 at 10.11.44 AM
Screen Shot 2021-01-11 at 10.11.44 AM742×437 57.2 KB

For the KeepX, I set a wide range for all datasets as the demand from my collaborator. This results in 27,204,300 models. To accelerate the process, I ran this script on a server using 100 CPUs. But I got this error. I am not sure how to set up “cpus” and “nrepeat” to run DIABLO in this situation.

Thank you very much!

Best,
Yuntao Yang

2

Hi @Yuntao_Yang ,

Thanks for using mixOmics.

I understand that you wish to test every possible combination of test.keepX but as you can see this will lead to memory exhaustion due to the high number of models to test. I recommend you start with a coarse grid (e.g. c(1:10, seq(15, 95, 5), seq(100, 900, 50) for DEGs) to first find the right neighbourhood with the superior performance. You can then specify a fine grid in that neighbourhood.

Hope it helps,

Al

3

Hi AI,

Thank you for your help! I will follow your suggestion.

Best,
Yuntao