We have seen that a vector space can have many different bases and that we can change between them via matrix muliplication. But is there ever a practical need to change basis?

Choosing a good basis to work with is central to some of the popular compression algorithms for audio and images. For example, the wavelet basis is widey used in digital signal processing. Discrete Fast Fourier Transform is an efficient change-of-basis method for expressing a matrix in terms of the wavelet basis.

Changing basis is also frequently employed in statistical analysis. In the present age of big data, linear dimension reduction through Principal Component Analysis is a common technique.

We now illustrate the use of changing basis with a simple example. Consider the following table listing the number of hours per week Ann, Bob, Cam, Dan, and Eve spend on studying and their midterm marks for algebra, calculus, and statistics.

Name Hours Algebra Calculus Statistics Ann Bob Cam Dan Eve 3 8.5 6 5 7 45 100 75 65 85 56 100 80 72 88 48 81 66 60 72
In real life, we could have hundreds of students and many more subjects. But for the sake of illustration, let us pretend that the above table is too large to load into the computer for analysis. Since we want to analyse the data as a whole, having only a subset of records at any given time is not an option. What can we do?

Let us first express the data as tuples in $$\mathbb{R}^4$$ as follows:

 Name Data Ann Bob Cam Dan Eve $$\begin{bmatrix}3\\45\\56\\48\end{bmatrix}$$ $$\begin{bmatrix}8.5\\100\\100\\81\end{bmatrix}$$ $$\begin{bmatrix}6\\75\\80\\66\end{bmatrix}$$ $$\begin{bmatrix}5\\65\\72\\60\end{bmatrix}$$ $$\begin{bmatrix}5\\85\\88\\72\end{bmatrix}$$

So far, we have not reduced the amount of data. We have just rewritten the table in a different way. But now observe that if $$d = \begin{bmatrix} 1\\10\\8\\6\end{bmatrix}$$ and $$v = \begin{bmatrix} 0 \\ 15 \\ 32 \\ 30\end{bmatrix}$$, then Ann's tuple is given by $$v + 3d$$, Bob's by $$v+8.5d$$, Cam's by $$v + 6 d$$, Dan's by $$v + 5d$$, and Eve's by $$v+7d$$. Hence, the scalar multiples of $$d$$ capture all the variability of the data.

Suppose that $$B = \{v_1, v_2, v_3, d\}$$ is a basis for $$\mathbb{R}^4$$. We know that every vector in $$\mathbb{R}^4$$ can be written as a linear combination of $$v_1,v_2,v_3$$, and $$d$$. But what is special about the tuples in the above table is that if $$u$$ is any of these tuples, then the linear combination of $$v_1,v_2,v_3$$, and $$d$$ expressing $$u - v$$ will be of the form $$0 v_1 + 0 v_2 + 0 v_3 + \alpha d$$ for some nonzero $$\alpha$$. In essence, with respect to the basis $$B$$, only one out of the four dimensions is relevant!

Remark. In real life, we almost never have data like those above that permit a perfect fit using a simple model. We are more likely to have data that resemble the following instead:

Name Hours Algebra Calculus Statistics Ann Bob Cam Dan Eve 3 8.5 6 5 7 47 100 74 64 87 55 99 80 73 89 50 80 65 59 73

Even though we cannot use $$v$$ and $$d$$ as above to perfectly match the data, we can still get very close. For practical purposes, an approximation often is good enough. As George E.P. Box once wrote, “Essentially, all models are wrong, but some are useful.” The key to doing proper statistical analysis is coming up with an appropriate model for the data at hand and determine if it is a good approximation.

Exercise

Give an orthogonal basis for $$\mathbb{R}^4$$ that contains the vector $$d = \begin{bmatrix} 1 \\ 10 \\ 8 \\ 6\end{bmatrix}$$.