Unexpected correlations compared to expression lines in DIABLO CircosPlot

Hi,

I have carried out a DIABLO analysis integrating three blocks of variables against my outcome (0 = healthy, 1 = diseased).
However, the CircosPlot shows incongruent results when comparing expression lines and correlations between some pairs of variables.
For example, based on the expression lines (and confirmed by a univariate comparison beween groups), variable X is higher in group 0 than in group 1, while variable Y is higher in group 1 than in group 0. However, the correlation between these variables depicted as positive. When I extract the correlation matrix from the CircosPlot, the correlation between the two variables is indeed positive. Can you please explain how this is possible?

Thank you,

Giorgia

Hi @gc2416

Thank you for using mixOmics and letting us know about this potential problem.

Can you please send us the diablo object from your analysis along with the script you use for us to look into it?

You can click on this text to send us an email.
Alternatively, you can right-click on the above text and choose ‘Copy Email Address’

Thanks

Al

Hi @gc2416,

Thanks for sending through the code and the data.

Please note that the degree to which the multivariate correlations would agree with the univariate ones would depend on the covariance structure between the blocks. They would generally agree but not in every case (see heatmaps of univariate vs multivariate correlations below). Additionally, looking more closely into your case, the ICNs block has a number of samples with completely missing values (see below). This would also lead to sub-optimal integration. Also, some metabolites are consistently missing across specific samples. In this case that the missing values are not random, you can look into imputation to see if you see any improvements (see mixOmics’ ?impute.nipals function or impute::impute.knn).

The following code goes through some of the points discussed.

Hope it helps

Al

library(mixOmics)
## change this to your own data path
load('/Users/alabadi/Projects/dev/R/_work/mixOmics/mixOmics_ajabadi/mixOmics_ajabadi/buildignore/devel/circosplot/MyDiablo_data.RData')

## get correlation matrix
corMat <-  circosPlot(MyResult.diablo, cutoff = 0.4)

## get univariate correlations for pairwise blocks
X.merge <- Reduce(cbind, MyResult.diablo$X)
univariate.cors <- cor(X.merge, use = 'pairwise.complete.obs')
## order features based on corMat
univariate.cors <- univariate.cors[rownames(corMat), colnames(corMat)]

## heatmaps generally agree but there are exceptions especially for low correlations
pheatmap::pheatmap(univariate.cors, cluster_rows = FALSE, cluster_cols = FALSE, show_rownames = FALSE, show_colnames = FALSE)

pheatmap::pheatmap(corMat, cluster_rows = FALSE, cluster_cols = FALSE, show_rownames = FALSE, show_colnames = FALSE)

## some samples are completely missing in ICNs (proportion of NA is 1)
# proportion of missing values for each feature in each block
lapply(MyResult.diablo$X, function(block) {
    n_features <- ncol(block)
    rowSums(is.na(block)) / n_features
})
#> $Metabolites
#>         1         2         3         4         5         6         7         8 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>         9        10        11        12        13        14        15        16 
#> 0.3529412 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>        17        18        19        20        21        22        23        24 
#> 0.0000000 0.3529412 0.0000000 0.0000000 0.0000000 0.0000000 0.3529412 0.0000000 
#>        25        26        27        28        29        30        31        32 
#> 0.3529412 0.0000000 0.0000000 0.3529412 0.0000000 0.0000000 0.0000000 0.0000000 
#>        33        34        35        36        37        38        39        40 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>        41        42        43        44        45        46        47        48 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>        49        50        51        52        53        54        55        56 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>        57        58        59        60        61        62        63        64 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>        65        66        67        68        69        70        71        72 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>        73        74        75        76        77        78        79        80 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.3529412 0.0000000 0.0000000 0.0000000 
#>        81        82        83        84        85        86        87        88 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>        89        90        91        92        93        94        95        96 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>        97        98        99       100       101       102       103       104 
#> 0.0000000 0.0000000 0.0000000 0.3529412 0.0000000 0.0000000 0.0000000 0.0000000 
#>       105       106       107       108       109       110       111       112 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>       113       114       115       116       117       118       119       120 
#> 0.0000000 0.0000000 0.3529412 0.3529412 0.0000000 0.0000000 0.0000000 0.0000000 
#>       121       122       123       124       125       126       127       128 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>       129       130       131       132       133       134       135       136 
#> 0.0000000 0.3529412 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>       137       138       139       140       141       142       143       144 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.3529412 
#>       145       146       147       148       149       150       151       152 
#> 0.3529412 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>       153       154       155       156       157       158       159       160 
#> 0.0000000 0.3529412 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>       161       162       163       164       165       166       167       168 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>       169       170       171       172       173       174       175       176 
#> 0.3529412 0.0000000 0.0000000 0.3529412 0.0000000 0.0000000 0.0000000 0.0000000 
#>       177       178       179       180       181       182       183       184 
#> 0.0000000 0.0000000 0.3529412 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>       185       186       187       188       189       190       191       192 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>       193       194       195       196       197       198       199       200 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.3529412 
#>       201       202       203       204       205       206       207       208 
#> 0.3529412 0.3529412 0.3529412 0.0000000 0.3529412 0.3529412 0.0000000 0.3529412 
#>       209       210       211       212       213       214       215       216 
#> 0.0000000 0.3529412 0.3529412 0.3529412 0.3529412 0.3529412 0.3529412 0.0000000 
#>       217       218       219       220       221       222       223       224 
#> 0.0000000 0.0000000 0.3529412 0.3529412 0.0000000 0.0000000 0.0000000 0.0000000 
#>       225       226       227       228       229       230       231       232 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.3529412 0.0000000 0.0000000 
#>       233       234       235       236       237       238       239       240 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.3529412 0.0000000 
#>       241       242       243       244       245       246       247       248 
#> 0.3529412 0.0000000 0.0000000 0.0000000 0.0000000 0.3529412 0.0000000 0.3529412 
#>       249       250       251       252       253       254       255       256 
#> 0.0000000 0.3529412 0.3529412 0.0000000 0.0000000 0.0000000 0.0000000 0.3529412 
#>       257       258       259       260       261       262       263       264 
#> 0.3529412 0.3529412 0.0000000 0.0000000 0.0000000 0.3529412 0.0000000 0.0000000 
#>       265       266       267       268       269       270       271       272 
#> 0.3529412 0.0000000 0.0000000 0.0000000 0.3529412 0.0000000 0.0000000 0.0000000 
#>       273       274       275       276       277       278       279       280 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>       281       282       283       284       285       286       287       288 
#> 0.0000000 0.3529412 0.0000000 0.0000000 0.3529412 0.3529412 0.0000000 0.0000000 
#>       289       290       291       292       293       294       295       296 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>       297       298       299       300       301       302       303       304 
#> 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 0.0000000 
#>       305       306       307 
#> 0.3529412 0.0000000 0.3529412 
#> 
#> $ICNs
#>   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17  18  19  20 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   1   0 
#>  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40 
#>   0   0   1   0   1   0   0   1   1   0   0   0   0   1   1   0   1   0   0   1 
#>  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60 
#>   0   0   1   0   1   0   0   1   0   0   0   0   0   0   0   0   0   0   1   0 
#>  61  62  63  64  65  66  67  68  69  70  71  72  73  74  75  76  77  78  79  80 
#>   0   1   0   1   0   0   0   0   1   0   0   0   0   0   0   0   0   1   0   1 
#>  81  82  83  84  85  86  87  88  89  90  91  92  93  94  95  96  97  98  99 100 
#>   1   1   0   0   1   0   0   0   0   0   0   0   0   1   1   1   0   0   1   0 
#> 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   1 
#> 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 
#>   0   0   0   0   1   1   0   0   0   1   0   0   0   0   0   1   0   0   0   0 
#> 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   1   0   1   0   0 
#> 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 
#>   0   1   0   0   0   0   0   0   0   1   1   0   0   0   0   0   0   0   0   1 
#> 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   1   0   1   1   0 
#> 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 
#>   0   0   0   1   1   0   0   0   0   1   0   1   0   0   1   1   0   1   0   0 
#> 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 
#>   0   0   0   0   0   0   1   0   1   0   1   0   0   1   0   1   0   1   0   1 
#> 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 
#>   0   1   1   0   0   1   0   0   1   0   0   0   1   0   0   1   0   0   0   0 
#> 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 
#>   0   1   0   0   0   0   0   0   0   0   0   1   0   1   0   0   1   0   0   1 
#> 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 
#>   1   0   0   0   0   0   1   0   0   0   0   0   0   0   1   0   0   0   0   0 
#> 301 302 303 304 305 306 307 
#>   0   0   0   0   0   1   1 
#> 
#> $`Inflammatory markers`
#>   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17  18  19  20 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 
#>  21  22  23  24  25  26  27  28  29  30  31  32  33  34  35  36  37  38  39  40 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 
#>  41  42  43  44  45  46  47  48  49  50  51  52  53  54  55  56  57  58  59  60 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 
#>  61  62  63  64  65  66  67  68  69  70  71  72  73  74  75  76  77  78  79  80 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 
#>  81  82  83  84  85  86  87  88  89  90  91  92  93  94  95  96  97  98  99 100 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 
#> 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 
#> 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 
#>   0   0   0   0   0   1   0   0   0   0   0   0   0   0   0   0   0   0   0   0 
#> 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 
#> 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 
#> 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 
#> 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 
#> 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 
#> 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 
#> 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 
#> 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 
#>   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0   0 
#> 301 302 303 304 305 306 307 
#>   0   0   0   0   0   0   0

Created on 2021-01-07 by the reprex package (v0.3.0)

Hi Al,

Thank you so much for such a detailed explanation!

Can I just clarify how the similarity matrix (i.e. the multivariate correlations) is calculated please? I’ve read that “the values in the similarity matrix are computed as the correlation between the two types of projected variables onto the space spanned by the first components retained in the analysis” (González et al., 2012). Does it mean that the correlation is computed on the weight (i.e. loading) of each variable on principal component 1 rather than between the variables themselves (as for univariate correlations)? If this was the case, there would be only 1 loading value for each variable, which would make it impossible to calculate a correlation between variable pairs. Can you please explain?

As for the multi-block correlation generated with the function plotDiablo() shown below, how is the correlation among blocks calculated? Is it a Pearson correlation between each pair of variates? And in the resulting plot, I suppose that the scores for each X block are plotted on a common variate?

Thanks,

Giorgia

Hi @gc2416,

Basically, for every block and for a given component:

  1. The correlation between original variables and the derived variates are calculated as a vector cord
  2. For a given pair of blocks i, j (i = j allowed), the outer product (cord_icord_j) of all the correlations calculated at step 1 is calculated which gives a feature by feature similarity matrix. This ensures a given pair of features would only be considered highly correlated if they both are highly correlated with the corresponding derived components of their respective blocks.
  3. All these values are merged across blocks to create a matrix including all pairwise similarity measures.

You can see the implementation at https://github.com/mixOmicsTeam/mixOmics/blob/master/R/circosPlot.R#L190-L202

As you would see in ?plotDiablo:
The lower triangular panel indicated the Pearson's correlation coefficient, the upper triangular panel the scatter plot.

I’m not sure if I got your question right but here’s what should address your query. For a given component, plotDiablo creates a variate by variate scatter plot for all pairs of blocks. This means for any given block we have only one vector of values shown (for the given component) against other variates.

Hope it helps.

Al