AUC for DIABLO object

Bugs
1

Hello,

I am using DIABLO to integrate transcriptomics and proteomics in order to identify a signature made up of ~2 transcripts and ~2 genes.

I have identified the signature using block.splsda, and now I would like to use leave one out cross validation to find out the AUC of the combined signature (genes and proteins).

I have used the perf function, however it will not provide the AUCs. This is the code I have ran:
perf.null <- perf(diablo.res.null, validation = ‘loo’, auc = TRUE,
nrepeat = 2, dist = ‘mahalanobis.dist’)

When I do perf.null$auc, auc is empty.
Is there an alternative way to do cross validation and then obtaining the combined cross-validated AUC?

Many thanks in advance,
Heather Jackson

2

Hi Heather,
Once you have your parameters keepX etc, run a full block.splsda on the whole data set and then:
auroc(object).

However, we won’t perform cross validation in this case, so we will follow up on this perf() code in the meantime.
Note: if you do leave-one-out cross-validation, you do not need to repeat the CV, only onces covers all possibilities!

Kim-Anh

3

Hi Kim-Anh,

Thanks a lot for your reply, and for pointing out about leave-one-out cross validation!

So is it only possible to see the AUC for each block individually, rather than the combined AUC?

Thanks
Heather

5

Hi @hj4817,

You can now install the latest version in which the perf function calculates the combined AUC (averaged across all blocks) for each component. Also, feel free to use the cpus argument for faster computation.

Please let us know if you run into any issues.

Best wishes,

Al

6

Dear Al, Kim-Anh,

I am trying to run the auroc on the DIABLO perf data, but it is giving the following error:

auroc(perf.diablo.final)
Error in UseMethod(“auroc”) :
no applicable method for ‘auroc’ applied to an object of class “perf.sgccda.mthd”

I am running the block.splsda function, resulting in my diablo.final object:

class(diablo.final)
[1] “block.splsda” “block.spls” “sgccda” “sgcca” “DA”

When I run the perf() function, it apparently uses the sgccda class in stead of the block.splsda. I am not sure whether that is the reason why it is not able to run the auroc() function… so far I have not able to figure out why is does not use the block.splsda and how to fix it:

class(perf.diablo.final)
[1] “perf.sgccda.mthd”

Any ideas?

Best, Lisette

7

Hi @LKogelman,

The auroc function can be applied to the diablo object directly. It cannot be applied to the perf object. You essentially use the perf function to choose the number of components and then evaluate the final diablo model using auroc. That is:

auroc.sgccda(diablo.final)

Let me know if you have further questions.

Al

8

HI @aljabadi

Thanks for your reply!
Ok, then I misunderstood the line you wrote above " You can now install the latest version in which the perf function calculates the combined AUC (averaged across all blocks) for each component."

How can I get the combined AUC for each component? That is the point where I understood I had to run the auroc after the perf function. I unfortunately cannot find the combined AUC of ‘diablo.final’ when using the auroc function.

Best,
Lisette

1 Like
9

Hi @LKogelman,
You can use auc = TRUE with perf to calculate the average AUC. See example below.

data(nutrimouse)
Y = nutrimouse$diet
data = list(gene = nutrimouse$gene, lipid = nutrimouse$lipid)
design = matrix(c(0,1,1,1,0,1,1,1,0), ncol = 3, nrow = 3, byrow = TRUE)
nutrimouse.sgccda <- block.splsda(X=data,
                                  Y = Y,
                                  design = design,
                                  keepX = list(gene=c(10,10), lipid=c(15,15)),
                                  ncomp = 2,
                                  scheme = "horst")

perf.res <- perf(nutrimouse.sgccda, auc = TRUE, folds = 3, nrepeat = 3)
perf.res$auc

Please let me know if you have further questions.

Best,

Al

2 Likes
10

Hi @aljabadi

In the above example, how can one plot the result of perf.res$auc?

Thanks,

Ramiro

11

We have the AUROC function.

12

Hi @aljabadi ,

Thanks for the tweak to compute the combined AUC per component. But the auroc plot function still plots per block per component?

I am currently using mixOmics_6.15.1. I tried looking into the auroc function but the current parameters does not seem to allow plotting those combined AUC. Is something out there that I might be missing?

Thanks again for your amazing suite of methods in mixOmics.