Matricies A and B are not the same size. If they were the same size (both 3x3) this would imply that A and B were similar matricies. In particular they would have similar diagonalization/Jordan form and have identical eigenvalues.
The eigenvalues of A are -2, 2, and 3. The eigenvalues of B are 2 and 3. The eigenvalues of B are a subset of A. Pad out B to create B' including the third eigenvalue -2:
[ 0 3 0 ]
B' = [-2 5 0 ]
[ 0 0 -2 ]
Now A and B' are similar matricies. They can both be diagonalized with the same diagonal matrix D. Let A = P*D*P^-1 and B' = Q*D*Q^-1. Then one diagonalization is:
[ 2 0 0 ] [ 14 2 2 ] [ 3 1 0 ]
D = [ 0 3 0 ] P = [ -9 -1 -1 ] Q = [ 2 1 0 ]
[ 0 0 -2 ] [ 1 0 1 ] [ 0 0 1 ]
Rearrange the equations for diagonalization to yield D = P^-1*A*P = Q^-1*B'*Q. Left multiply each side by P and right multiply each side by Q^-1 to get A*P*Q^-1 = P*Q^-1*B'. Then C'=P*Q^-1 is a matrix which solves A*C' = C'*B'.
[ 10 -8 2 ]
C' = [ -7 6 -1 ]
[ 1 -1 1 ]
Finally C' can be reduced to create C be removing the rightmost column (C will have rank 2 because C' has rank 3 from being invertible):
[ 10 -8 ]
C = [ -7 6 ]
[ 1 -1 ]
A*C = C*B can be verified by direct multiplication:
[ 16 -10 ]
A*C = C*B = [-12 9 ]
[ 2 -2 ]
|