<table width = 900 border = 0><tr><td>
# <b>NMST539 | Lab Session 10</b>

## <b>Canonical Corelation Analysis</b>
###  (application in R)

### <font color = "#880000"><b>LS 2017 | Tuesday 16/05/2017</b></font>
###### <a href = "cvicenie2017_11.Rmd">Rmd file (UTF8 encoding)</a>

**************
The R-software is available for download from the website: <a href = "https://www.r-project.org">https://www.r-project.org</a>

A user-friendly interface (one of many): <a href = "https://www.rstudio.com">RStudio</a>.<br><br>

<b>Manuals and introduction into <i>R</i>
(in Czech or English):</b><br>
    
<ul>
<li>Bína, V., Komárek, A. a Komárková, L.: <i>Jak na jazyk R.</i>  (<a href = "NMFM301/Rmanual2.pdf">PDF súbor</a>)</li>
<li>Komárek, A.: <i>Základy práce s R.</i>  (<a href = "http://msekce.karlin.mff.cuni.cz/~komarek/vyuka/dataRko/Rmanual1.pdf">PDF súbor</a>)</li>
<li>Kulich, M.: <i>Velmi stručný úvod do R.</i> (<a href = "NMFM301/uvodr.pdf" >PDF súbor</a>)</li>
<li>De Vries, A. a Meys, J.: <i>R for Dummies.</i> (ISBN-13: 978-1119055808) </li>  	
</ul>
**************

<br><br>

### <font color = "#880000"><b>Canonical Correlation Analysis in R</b></font>

The canonical correlation analysis (CCA) is a multidimensional exploratory statistical method
which (roughly) operates on the same principle as the principal component analysis (PCA). However, unlike the PCA where one tries to explore the common variability within the given data set, the canonical correlation is used to explore the mutual 
variability (correlations) between two different sets of quantitative variables observed on the same experimental units. Thus, the PCA method deals with one dataset only and it tries to to reduce the overall dimensionality of the dataset using some linear combination of the initial variables with the maximum variace possible
while the CCA approach searhes for some linear combinations of the initial variables in order to achieve maximum correlations instead. <br><br>

For instance, for two multivariate random vectors $\boldsymbol{X}$ and $\boldsymbol{Y}$ we assume that the common variance-covariance matrix of $(\boldsymbol{X}, \boldsymbol{Y})^\top$ is of the form 
$$
Var \left(\begin{array}{c}\boldsymbol{X}\\ \boldsymbol{Y}\end{array}\right) = \left(\begin{array}{cc} \Sigma_{XX} & \Sigma_{XY}\\ \Sigma_{YX} & \Sigma_{YY} \end{array}\right).
$$

If $\boldsymbol{X}$ is a $p$ dimensional random vector and $\boldsymbol{Y}$ is a $q$ dimensional random vecton, then the corresponding correlation of any linear combination of $\boldsymbol{X}$ and $\boldsymbol{Y}$ in terms
of $\boldsymbol{a}^\top \boldsymbol{X}$ and $\boldsymbol{b}^\top\boldsymbol{Y}$, for $\boldsymbol{a} \in \mathbb{R}^p$ and $\boldsymbol{b} \in \mathbb{R}^q$ can be expressed as 
$$
Cor(\boldsymbol{a}^\top \boldsymbol{X}, \boldsymbol{b}^\top\boldsymbol{Y}) = \boldsymbol{a}^\top \Sigma_{XY} \boldsymbol{b}.
$$

Therefore, the canonical correlations analysis searches for such vectors $\boldsymbol{a} \in \mathbb{R}^p$ and $\boldsymbol{b} \in \mathbb{R}^q$, that the previous correlation is as large as possible. However, 
the canonical correlation analysis does not search for just one pair of the canonical variables $\boldsymbol{a}^\top \boldsymbol{X}$ and $\boldsymbol{b}^\top\boldsymbol{Y}$ but instead, it defines all pairs (up to $min(p, q)$), 
such that all canonical covariates within each dataset are uncorrelated among each other. <br><br>

In the R statistical software there is function `cancor()` available under the standard instalation. 
Some additional packages can be downloaded and installed as well (mainly the R packages 'CCA' and 'vegan'). 

In the following we will consider the same dataset we have already worked with before. The data consists 
of two datasets where each dataset represents measurements in some specific (64-65) river localities in the Czech republic. The first dataset contains measurements of biologial metrics for each locality (17 different metrics and taxons) and the secon dataset contains measurements on chemical concentrations and values at the same localities (7 covariates recored). 

The idea is to somehow relate both dataset in order to explain what bio metrics can correlate which which chemical concentrations. One way of answering this is to apply the canonical correlation approach explained below. 


```{r, warning = F, message = F}
rm(list = ls())
bioData <- read.csv("http://msekce.karlin.mff.cuni.cz/~maciak/NMST539/bioData.csv", header = T)
chemData <- read.csv("http://msekce.karlin.mff.cuni.cz/~maciak/NMST539/chemData.csv", header = T)

head(bioData)
head(chemData)
```
<br><br>

Again, there is one locality which is missing the dataset of chemical measurements. We identify this locality and we do not consider it for further analysis. Both datasets are alligned with respect to he locality names - the first column in each dataset. 


```{r, warning = F, message = F}
ind <- match(chemData[,1], bioData[,1])
data <- data.frame(bioData[ind, ], chemData[, 2:8])
```


```{r}
X <- data[,2:9]
Y <- data[,19:25]
```

A nice graphical tool to view both datasets with respect to their overall (within and between) correlation structure is avialable within the R package 'CCA' (Canonical Correlation Analysis). The packages needs to be firstly installed (`install.packages("CCA")`) and later one can use functions `matcor()` and `img.matcor()` to graphically display the corration matrix 

$$ cor(\mathcal{X}, \mathcal{Y}) = \left(\begin{align}\Sigma_{XX} & \Sigma_{XY}\\\Sigma_{YX} & \Sigma_{YY}\end{align}\right)$$.

```{r, warning=F, message = F, fig.width=10, fig.height=8}
library("CCA")
correl <- matcor(X, Y )
img.matcor(correl, type = 2)
```


Next we can directly use `cancor()` function to obtain canonical correlations for the datasets $\mathcal{X}$ and $\mathcal{Y}$. Another option is to use function `cc()` (from the R package 'CCA'). Note, the the results are equivalent although not identical. 

```{r}
cc1 <- cancor(X, Y)  ### function from standard R instalation
cc2 <- cc(X, Y)      ### function for the R package 'CCA'
```

The canonical correlations are the same for both approaches (R commands used) 

```{r}
cc1$cor  ### function from standard R instalation
cc2$cor  ### function for the R package 'CCA'
```


The canonical covariates can be easily reconstructred as follows: 
```{r}
CanX <- as.matrix(X) %*% cc1$xcoef
CanY <- as.matrix(Y) %*% cc1$ycoef

cor(CanX, CanY)
```

Which corresponds with the output provided by the function `cancor()` and `cc()`above (see also the picture below
for a visual comparison).<br><br>



```{r, warning=F, message = F, fig.width=10, fig.height=8}
par(mfrow = c(1,2))
barplot(cc1$cor, main = "Canonical correlations for 'cancor()'", col = "gray")
barplot(cc2$cor, main = "Canonical correlations for 'cc()'", col = "gray")
```


However, the actual estimated linear combinations of the original covariates in both datasets $\mathcal{X}$ and $\mathcal{Y}$ are different. Compare the following: 

```{r}
cc1$xcoef  ### function from standard R instalation
cc2$xcoef  ### function for the R package 'CCA'

cc1$ycoef
cc2$ycoef
```

Using the 'CCA' package one can also take an advantage of nice graphical tools which are available for
the canonical correlation analysis. The idea is to display maximized correlations between transformed variables of the dataset $\mathcal{X}$ and the dataset $\mathcal{Y}$. 

```{r, warning=F, message = F, fig.width=10, fig.height=8}
plt.cc(cc2, var.label = TRUE, ind.names = data[,1])
```


#### <font color = "#880000">Questions</font>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />
 <ul>
 <li>How would you interpret the left hand side of the figure?</li>
 
 <li>What is the relationship between the subfigure on the left hand side and the subfigure on the 
 right hand side? </li>
 
 <li>What would be your conclussion about the two available dataset and their mutual relationship? </li>
 </ul>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />
<br><br>

A usefull extension to the classical canonical correlation approach is the regularized version - so called regularized canonical correlation analysis. It is mean for scenarios where the number of parameters $p \in \mathbb{N}$ is greater than the sample size $n \in \mathbb{n}$ (usually $p \gg n$). 
There is R function `rcc()` available in the R package 'CCA'. <br><br>

Another option how to perform canonical correlation in R is to install an additional R package - package 'vegan' (use `install.packages("vegan")` for instalation) and to apply function `cca()`.  Use the help session in R for more details about the function itself or the corresponding package respectively.




```{r, warning=F, message = F}
library(vegan)
cc3 <- cca(X, Y)
```

And the corresponding graphical summary given by `plot()` function.

```{r, warning=F, message = F, fig.width=10, fig.height=8}
plot(cc3, scaling = 1)
```


#### <font color = "#880000">To Do</font>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />
 <ul>
 <li>Use this markdown and consider (maybe) the whole dataset on biologial measurements. Compare how 
 much the results changes. </li>
 
 <li>Consider a lower dimensional profile and try to refit the whole canonical correlation analysis on
 a lower dimensional datasets (e.g. two covariates for bio data and two covariates for chemical data). </li>
 
 <li>Try to manually reconstruct the results from the above by using the correponding canonical correlation theory form the lecture. </li>
 
 
 </ul>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />
<br><br>




<br><br>
<hr size="3" width="100%" noshade style="color:#440000" align="left" />
<hr size="3" width="100%" noshade style="color:#440000" align="left" /><br><br> 




## <font color = "#880000"><b>Homework Assignment</b></font>
### (Deadline: last Tuesday / 23.05.2007)

Use the command below to instal the `SMSdata` packages with various multivariate datasets: 

```{r, eval = F}
install.packages(pkgs="http://www.karlin.mff.cuni.cz/~hlavka/sms2/SMSdata_1.0.tar.gz", repos=NULL, type="source")
```

Chose one dataset of your choise from the list of all available datasets in the package:

```{r, eval = F}
data(package = "SMSdata") 
```
 
 There are 21 different datasets and you can load each of them by typing and running 
`data(dataset_name)`, for instance:

```{r, eval = F}
data(plasma) 
```

<ul>
<li>Choose one dataset and use some judgement to split the data into two disjoint parts. Alternatively, 
you could also consider two different datasets however, they should be recorded on the same units. 
Apply the canonical correlation analysi. </li>

<li>Finilize your report and make it ready for the final submission. The submission deadline 
is <font color = "red">Monday, 22/05/2017, 23:59</li>. The report should be submitted by emial 
to (at least) one of the following addresses: 
<ul>
<li>maciak AT karlin.mff.cuni.cz</li>
<li>maciak AT ualberta.ca</li>
</ul>
</ul>





</td></tr></table>
<br><br><br>