# Greg Borenstein's NoMM Page

## Transforms: Wavelets, PCA, ICA

12.3) Consider a measurement of a three-component vector x, with x1 and x2 being drawn independently from a Gaussian distribution with 0 mean and unit variance, and x3 = x1 + x2.

a) Analytically calculate the covariance matrix of x

b) What are the eigenvalues?

C) Numerically verify these results by drawing a data set from the distribution and computing the covariance matrix and eigenvalues.

Code here: pca.py

``` import numpy as np n = 100000 x1 = np.random.normal(0,1,n) x2 = np.random.normal(0,1,n) x3 = x1+x2 X = [x1,x2,x3] c = np.cov(X) print(c) # Covariance matrix: # [[ 1.00552305 -0.00540942 1.00011363] # [-0.00540942 1.00327819 0.99786877] # [ 1.00011363 0.99786877 1.9979824 ]] e = np.linalg.eig(c) print(e[0]) # Eigenvalues: # [ 2.99277382e+00 1.00136031e+00 1.76529821e-15] ```

D) Numerically ﬁnd the eigenvectors of the covariance matrix, and use them to construct a transformation to a new set of variables y that have a diagonal covariance matrix with no zero eigenvalues. Verify this on the data set.

Continuing the excerpt of pca.py above:

``` print(e[1]) # Eigenvectors (col major): # [[-0.40445603 0.70928273 0.57735027] # [-0.41202885 -0.70491056 0.57735027] # [-0.81648487 0.00437217 -0.57735027]] # Cov(y) = M * Cov(x) * M^T cy = np.dot(np.dot(e[1].T,c), e[1].T) print(cy) # # [[ 2.61908171 0.80509741 -0.50471862] # [ 0.80509741 0.98241759 0.30643185] # [-0.50471862 0.30643185 0.38716296]] ```

12.4) Generate pairs of uniform random variables {s1,s2} with each component contained in [0,1].

a) Plot these data.

Code here: ica.py.

``` import numpy as np import matplotlib.pyplot as plt n = 100 s1 = np.random.uniform(0,1,n) s2 = np.random.uniform(0,1,n) plt.scatter(s1,s2) plt.show() ```

b) Mix them (x = A*s) with a square matrix A = [[1 2] [3 1]] and plot.

Continuing from ica.py...

``` S = np.column_stack([s1, s2]) A = [[1,2],[3,1]] mix = np.dot(S,A) plt.scatter(mix[:,0], mix[:,1]) plt.show() ```

c) Make x zero mean, diagonalize with unit variance, and plot