Let \(A = \begin{bmatrix} 1 & 2 \\ 2 & 4\end{bmatrix}\).
The answer is “No”.
Adding \(-2\) times row 1 of the matrix \([A \mid I]\) to row 2 gives
\(\begin{bmatrix} 1 & 2 & 1 & 0\\ 0 & 0 & -2 & 1\end{bmatrix}\).
Since the matrix is in reduced row-echelon form and the first two
columns do not give the identity matrix, \(A\) has no inverse.