Linear transformation view

In \(Ax=b\), the left-hand side is a tuple whose entries are of the form \(a_1x_1+\cdots +a_nx_n\). For example, when \(A = \begin{bmatrix} 1 & 2 & 3\\ 4 & 5 & 6\end{bmatrix}\), \(Ax\) is the tuple \(\begin{bmatrix} x_1 + 2x_2 + 3x_3 \\ 4x_1 + 5x_2 + 6x_3\end{bmatrix}\). Here, both entries are of the form \(a_1 x_1 + a_2 x_2 + a_3 x_3\) for some real numbers \(a_1,a_2,a_3\).

The left-hand side could be interpreted as the output of some mapping \(T\) with input \(x\). In this case, \(T\) is said to be linear because every entry of the tuple it returns is a linear combination of the entries of \(x\).

The actual definition of a linear mapping (or linear transformation) will be given in the future. For now, it is sufficient to know that any linear transformation \(T\) that takes an \(n\)-tuple as input and outputs an \(m\)-tuple can be written as \(T(x)=Ax\) for some matrix \(A\) with \(m\) rows and \(n\) columns. Conversely, given a matrix \(A\) with \(m\) rows and \(n\) columns, \(T(x) = Ax\) is a linear transformation that accepts an \(n\)-tuple as input and outputs an \(m\)-tuple.

Examples

1. Let \(T\) be a mapping given by \(T\left(\begin{bmatrix} x_1 \\ x_2\end{bmatrix} \right)= \begin{bmatrix} x_1 + 4x_2\\ 2x_1 + 5x_2\\ 3x_1 + 6x_2 \end{bmatrix}\). So \(T\) is a linear transformation that accepts 2-tuples as input and outputs 3-tuples. It can be written as \(T(x) = Ax\) with \(A = \begin{bmatrix} 1 & 4\\ 2 & 5\\ 3 & 6 \end{bmatrix}.\)

2. Let \(T\) be a mapping given by \(T\left(\begin{bmatrix} x_1 \\ x_2 \\ x_3\end{bmatrix} \right)= \begin{bmatrix} x_3 \\ x_1\\ x_2 \end{bmatrix}\). More verbosely, \(T\left(\begin{bmatrix} x_1 \\ x_2 \\ x_3\end{bmatrix} \right)= \begin{bmatrix} 0x_1 + 0x_2 + 1x_3 \\ 1x_1 + 0x_2 + 0x_3\\ 0x_1 + 1x_2 + 0x_3 \end{bmatrix}\). So \(T\) can be given as \(T(x) = Ax\) with \(A = \begin{bmatrix} 0 & 0 & 1\\ 1 & 0 & 0\\ 0 & 1 & 0 \end{bmatrix}.\)

A new view

Solving \(Ax = b\) can be interpreted as finding which inputs to the linear transformation \(T\) given by \(T(x) = Ax\) lead to \(b\) as output. This is precisely the question of determining the pre-image of \(b\) under \(T\).

But what does one gain by taking this view? It turns out that row reduction can be viewed as applying a sequence of linear transformations to both sides of \(Ax = b\), which involves the composition of linear transformations.

Composition of linear transformations

Consider the system given by \(S(x)=b\) where \(S(x) = Ax\) for some \(p\times n\) matrix \(A\) (i.e. \(A\) has \(p\) rows and \(n\) columns). Let \(T\) be a linear transformation that accepts \(p\)-tuples as inputs. Applying \(T\) to both sides of the system gives \(T(S(x)) = T(b)\). It happens that this system is also linear.

Example

Suppose that \(S(x) = Ax\) where \(A=\begin{bmatrix} 1&2&3\\4&5&6\end{bmatrix}\) and \(x=\begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix}\), and \(T(y)=By\) where \(B = \begin{bmatrix} 7 & -1 \\ -2 & 1\end{bmatrix}\) and \(y=\begin{bmatrix}y_1\\y_2\end{bmatrix}\). What does \(T(S(x))\) look like?

First, note that \(S\left(\begin{bmatrix} x_1\\x_2\\x_3 \end{bmatrix}\right) = \begin{bmatrix} x_1 + 2x_2 + 3x_3\\4x_1+5x_2+6x_3\end{bmatrix}\) and that \(T\left(\begin{bmatrix} y_1\\y_2 \end{bmatrix}\right) = \begin{bmatrix} 7y_1 - y_2 \\-2y_1+y_2\end{bmatrix}\). Hence, \begin{eqnarray} T(S(x)) & = & T\left(\begin{bmatrix} x_1 + 2x_2 + 3x_3\\4x_1+5x_2+6x_3\end{bmatrix}\right) \\ & = & \begin{bmatrix} 7(x_1 + 2x_2 + 3x_3)+(-1)(4x_1+5x_2+6x_3) \\ -2(x_1 + 2x_2 + 3x_3) + (4x_1 + 5x_2 + 6x_3) \end{bmatrix} \\ & = & \begin{bmatrix} 3x_1 + 9x_2 + 15x_3 \\ 2x_1 + x_2 \end{bmatrix} \\ & = & Cx \end{eqnarray} where \(C = \begin{bmatrix} 3 & 9 & 15 \\ 2 & 1 & 0\end{bmatrix}\).

Thus, \(T(S(x)) = U(x)\) for some linear transformation \(U\). Written in terms of matrices, we have \(B(Ax) = Cx\).

One can easily verify that \(C = \begin{bmatrix} BA_1 & BA_2 & BA_3\end{bmatrix}\) where \(A_i\) denotes the \(i\)th column of \(A\). If we define the product \(BA\) as \(\begin{bmatrix} BA_1 & BA_2 & BA_3\end{bmatrix}\), then we can write \(B(Ax) = (BA)x\).

In general, composing linear transformations results in another linear transformation. The matrix that represents the resulting linear transformation can be expressed as a product of matrices once we have formally defined matrix multiplication.

Quick Quiz

Exercises

1. Let \(T\) be a linear transformation given by \(T\left(\begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix}\right) = \begin{bmatrix} 2x_1 - x_3 \\ x_2 + 3x_3 \end{bmatrix}\). Find a matrix \(A\) such that \(T(x) = Ax\).

   Note that \(\begin{bmatrix} 2x_1 - x_3 \\ x_2 + 3x_3 \end{bmatrix}\) can be rewritten as \(\left[\begin{array}{rcrcr} 2x_1 & + & 0x_2 & + & (-1) x_3 \\ 0x_1 & + & x_2 & + & 3x_3 \end{array}\right]\). So \(A = \begin{bmatrix} 2 & 0 & -1 \\ 0 & 1 & 3\end{bmatrix}\).

2. Let \(S(x) = Ax\) where \(A=\begin{bmatrix} 1&-1&0\\4&1&-2\end{bmatrix}\) and \(x=\begin{bmatrix}x_1\\x_2\\x_3\end{bmatrix}\), and \(T(y)=By\) where \(\begin{bmatrix} 3 & 1\end{bmatrix}\) and \(y=\begin{bmatrix}y_1\\y_2\end{bmatrix}\).

   1. Give the matrix \(C\) such that \(T(S(x)) = Cx\).

      \(C = \begin{bmatrix} 7 & -2 & -2\end{bmatrix}\).

   2. Let \(b = \begin{bmatrix} 1\\ 1\end{bmatrix}\). Find all solutions to the systems \(S(x) = b\) and \(T(S(x)) = T(b)\). Are the solution sets the same?

      The solution sets are not the same.

      The solutions to \(S(x) = b\) are given by \( \begin{bmatrix} x_1\\x_2\\x_3\end{bmatrix} = \begin{bmatrix} \frac{2}{5} \\ - \frac{3}{5} \\ 0\end{bmatrix} + s \begin{bmatrix} \frac{2}{5} \\ \frac{2}{5} \\ 1\end{bmatrix} \) where \(s \in \mathbb{R}\).

      To obtain the solutions to \(T(S(x)) = T(b)\), note that \(T(b) = 4\). Thus \(T(S(x) = T(b)\) is the equation \(7x_1 - 2x_2 - 2x_3 = 4\). The solutions are given by \( \begin{bmatrix} x_1\\x_2\\x_3\end{bmatrix} = \begin{bmatrix} \frac{4}{7} \\ 0 \\ 0\end{bmatrix} + s \begin{bmatrix} \frac{2}{7} \\ 1 \\ 0\end{bmatrix} + t \begin{bmatrix} \frac{2}{7} \\ 0 \\ 1 \end{bmatrix} \) where \(s, t \in \mathbb{R}\).

      Note that \( \begin{bmatrix} x_1\\x_2\\x_3\end{bmatrix} = \begin{bmatrix} \frac{4}{7} \\ 0 \\ 0\end{bmatrix}\) is a solution to \(T(S(x) = T(b)\) but not to \(S(x) = b\).