Hi everyone, I am currently working with the mixOmics package. I was wondering whether I could add a 95% confidence interval to the AUC value. I am using different statistical models and want to compare them to each other. For the other models, I get an AUC + 95% CI, so I would like to get this as well for the sPLS-DA I am performing.

I used the auroc() function to get the ROC curve and the AUC value, but I am missing a 95% CI. I can’t think of a way to get to that 95% CI that works.

I know that I can create AUC + 95% CI with the pROC package. However, then I do not understand how to use my sPLS-DA model in the roc() function of this package.

Is there anybody that could help me out? It would help a lot with my master thesis!

Thanks in advance!

Kind regards,

Annemieke

hi @Annemieke,

It’s a technical question! You can access the R code here: mixOmics/R/auroc.R at master · mixOmicsTeam/mixOmics · GitHub

Roughly, it happens at line 121 where you get the prediction `res.predict`

, and this is then use to get `statauc.res`

which is the result of the plot.

So if you are R savvy you might be able to play with this and extract what you need

(short answer is: I dont have a better quick fix for this)

Kim-Anh

Hi @kimanh.lecao ,

Thank you so much for your quick response!

I got it to work

Hello! Same problem here. Could you explain how you did it? I am trying to extract the CI from resulting ROC curves from a tuned sPLS-DA model. Thanks!

Hi @FGarrido!

I figured it out in a different way than described above. I tried to explain my code below! It is a bit of a long, weird way, but it works, so that is something I guess? And I checked whether it is correct with SPSS, so it should be alright.

I hope you understand the code below, let me know otherwise

Good to know: My dataset is called training, my groups are defined within the column DiagnosisCategory (so diagnosis category is the “y” in my model)

```
final_sPLSDA <- splsda(x, y, ncomp = optimal_ncomp, keepX = optimal_nvariables) #final model
prediction <- predict(final_sPLSDA, x, dist = "mahalanobis.dist") #predict how well the model works
```

I made a table in which the discriminant scores and the diagnosis category are included with the following code

```
DiscriminantScoresTable <- cbind(as.data.frame(training$DiagnosisCategory) , as.data.frame(prediction$predict[,,"dim3"])) #the number in dimensions depends on the number of components included in your model
```

```
ROC <- roc(DiscriminantScoresTable$`data$DiagnosisCategory`, DiscriminantScoresTraining$NT1) #NT1 is one of my groups. The group you fill in is set as the case (I compared narcolepsy with healthy controls, so NT1 were my cases)
ROC$auc #for the AUC value
ci.auc(ROC$auc) #for the 95% confidence interval
```

2 Likes

Thanks for the reply!

That is a very intelligent solution! I was also playing with the predict function, but I thought that CI could be performed without the pROC package. Thanks a lot, it works perfectly

Just to note: inside the “roc” function, the NT1 column is selected from another data frame? I used my original DiscriminantScoreTable, which works just fine (it returns the same values as the auroc function).

Ah yeah, I see it! I tried to make my code a bit more understandable to post here by adjusting some names, but I forgot to adjust that

Glad it works!

1 Like