From what is given, \(\sigma(1) = 3\), \(\sigma(2) = 1\), and \(\sigma(3) = 2\). Hence, \(A_{1\sigma(1)} A_{2\sigma(2)} A_{3\sigma(3)} = A_{1,3} A_{2,1} A_{3,2} = 4\cdot (-5) \cdot 2 = -40\).

We will compute the determinant using three different methods.

Method 1 (formula for \(3\times 3\) matrices)

The determinant is \( (-i)\cdot 2\cdot 1 + 1\cdot 0 \cdot (-1) + 0\cdot 1 \cdot 1 - 0\cdot 2\cdot (-1) - (-i)\cdot 0 \cdot 1 - 1\cdot 1 \cdot 1 = -1 -2i.\)

Method 2 (row reduction)

\[ \begin{array}{rcll} \left| \begin{array}{ccc} -i & 1 & 0 \\ 1 & 2 & 0 \\ -1 & 1 & 1 \end{array}\right | & = & -\left| \begin{array}{ccc} 1 & 2 & 0 \\-i & 1 & 0 \\-1 & 1 & 1 \end{array}\right | & (R_1 \leftrightarrow R_2) \\ & = & -\left| \begin{array}{ccc} 1 & 2 & 0 \\-i & 1 & 0 \\ 0 & 3 & 1 \end{array}\right | & (R_3 \leftarrow R_3 + R_1) \\ & = & -\left| \begin{array}{ccc} 1 & 2 & 0 \\ 0 & 1+2i & 0 \\ 0 & 3 & 1 \end{array}\right | & (R_2 \leftarrow R_2 + iR_1) \\ & = & -(1+2i)\left| \begin{array}{ccc} 1 & 2 & 0 \\ 0 & 1 & 0 \\ 0 & 3 & 1 \end{array}\right | & (R_2 \leftarrow \frac{1}{1+2i}R_2) \\ & = & (-1-2i)\left| \begin{array}{ccc} 1 & 2 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array}\right | & (R_3 \leftarrow R_3 - 3R_2) \\ & = & -1-2i. \end{array}\]

Method 3 (cofactor formula)

Note that there are many 0's in the third column. So using the cofactor expansion along column 3, we get that the determinant is \((-1)^{3+3}(1)\left | \begin{array}{cc} -i & 1 \\ 1 & 2\end{array} \right | = (-i)\cdot 2 - 1\cdot 1 = -1 - 2i\).

We will use the fact that \(A\) is singular if and only if \(\det(A) = 0\).

Note that in row 2, only the entry in the fourth column is nonzero. Hence, using the cofactor expansion along row 2, we see that \(\det(A)\) is simply \begin{eqnarray*} & & (-1)^{2+4} (-1) \left |\begin{array}{ccc} k & 1 & 1 \\ 1 & 0 & k \\ 1 & -1 & 2 \end{array} \right | \\ & = & -(k\cdot 0\cdot 2 + 1\cdot k \cdot 1 + 1\cdot 1\cdot(-1) - 1\cdot 0\cdot 1 - (-1)\cdot k\cdot k - 2\cdot 1\cdot 1) \\ & = & -(k - 1 + k^2 - 2) \\ & = & -k^2 -k + 3. \\ \end{eqnarray*} Hence, \(\det(A) = 0\) if and only if \(k = (-1\pm\sqrt{13})/2\). Hence, the values of \(k\) for which \(A\) is singular are \((-1+\sqrt{13})/2\) and \((-1-\sqrt{13})/2.\)

Consider the product of row \(i\) of \(A\) with column \(j\) of \(B\): \begin{eqnarray*} & & A_{i,1} C_{j,1} + A_{i,2} C_{j,2} + \cdots A_{i,n} C_{j,n} \\ & = & (-1)^{j+1}A_{i,1}\det(A(j\mid 1)) + (-1)^{j+2}A_{i,2}\det(A(j\mid 2)) + \cdots (-1)^{j+n}A_{i,n}\det(A(j\mid n)) \end{eqnarray*} If \(i = j\), then this sum is simply the cofactor formula for the determinant of \(A\) expanded along row \(i\) and thus equals \(\det(A)\). Thus the diagonal entries of \(AB\) are equal to \(\det(A)\).

Suppose that \(i \neq j\). Let \(\tilde{A}\) be obtained from \(A\) by adding row \(i\) to row \(j\). Using the cofactor formula expanding along row \(j\) of \(\tilde{A}\), we get \begin{eqnarray*} \det(\tilde{A}) & = & (-1)^{j+1}\tilde{A}_{j,1}\det(\tilde{A}(j\mid 1)) + (-1)^{j+2}\tilde{A}_{j,2}\det(\tilde{A}(j\mid 2)) + \cdots (-1)^{j+n}\tilde{A}_{j,n}\det(\tilde{A}(j\mid n)) \\ & = & (-1)^{j+1}(A_{j,1} + A_{i,1})\det(A(j\mid 1)) + (-1)^{j+2}(A_{j,2} + A_{i,2})\det(A(j\mid 2)) + \cdots (-1)^{j+n}(A_{j,n} + A_{i,n})\det(A(j\mid n)) \\ & = & (-1)^{j+1}A_{j,1}\det(A(j\mid 1)) + (-1)^{j+2}A_{j,2}\det(A(j\mid 2)) + \cdots (-1)^{j+n}A_{j,n}\det(A(j\mid n)) + \\ & & (-1)^{j+1}A_{i,1}\det(A(j\mid 1)) + (-1)^{j+2}A_{i,2}\det(A(j\mid 2)) + \cdots (-1)^{j+n}A_{i,n}\det(A(j\mid n)) \\ & = & \det(A) + (-1)^{j+1}(A_{j,1}+ A_{i,1})\det(A(j\mid 1)) + (-1)^{j+2}(A_{j,2} + A_{i,2})\det(A(j\mid 2)) + \cdots (-1)^{j+n}(A_{j,n} + A_{i,n})\det(A(j\mid n)) \end{eqnarray*} Since \(\det(\tilde{A}) = \det(A)\), we see that \((-1)^{j+1}A_{i,1}\det(A(j\mid 1)) + (-1)^{j+2}A_{i,2}\det(A(j\mid 2)) + \cdots (-1)^{j+n}A_{i,n}\det(A(j\mid n)) = 0\). Hence, the nondiagonal entries of \(AB\) are 0. Thus, \(AB = \det(A) I_n\) as desired.

Remark. Note that if \(\det(A) \neq 0\), then \(\frac{1}{\det(A)} B\) is the inverse of \(A\). Thus, the result gives a formula for the inverse matrix, if it exists.

We will try to write \(\det(A^{-2}B^\mathsf{T})\) in terms of \(\det(A)\) and \(\det(B)\). First, \(B\) is an upper triangular matrix. Hence, \(\det(B)\) is given by the product of the diagonal entries which is \(2i\). Then, \begin{eqnarray*} \det(A^{-2}B^\mathsf{T}) & = & \det(A^{-2})\det(B^\mathsf{T}) \\ & = & \det( (A^{-1})^2) \det(B) \\ & = & \det( A^{-1})^2 (2i) \\ & = & \left(\frac{1}{\det(A)}\right)^2 (2i) \\ & = & \left(\frac{1}{1-i}\right)^2 (2i) \\ & = & \frac{1}{-2i} (2i) \\ & = & -1. \end{eqnarray*}