You're doing a direct sum instead of a tensor product of the two vectors; as such, your input state is wrong.
A direct sum operates by concatenating the two vectors: $$ \psi \oplus \phi = \begin{pmatrix}\psi_1\\\psi_2\end{pmatrix} \oplus \begin{pmatrix}\phi_1\\\phi_2\end{pmatrix} = \begin{pmatrix} \begin{pmatrix}\psi_1\\\psi_2\end{pmatrix}\\ \begin{pmatrix}\phi_1\\\phi_2\end{pmatrix} \end{pmatrix} = \begin{pmatrix}\psi_1\\\psi_2\\\phi_1\\\phi_2\end{pmatrix} . $$ In a tensor product, on the other hand, you form a 'vector of vectors', whose entries are products of the components of the initial two vectors: $$ \psi \otimes \phi = \begin{pmatrix}\psi_1\\\psi_2\end{pmatrix} \otimes \begin{pmatrix}\phi_1\\\phi_2\end{pmatrix} = \begin{pmatrix} \psi_1\begin{pmatrix}\phi_1\\\phi_2\end{pmatrix}\\ \psi_2\begin{pmatrix}\phi_1\\\phi_2\end{pmatrix} \end{pmatrix} = \begin{pmatrix}\psi_1\phi_1\\\psi_1\phi_2\\\psi_2\phi_1\\\psi_2\phi_2\end{pmatrix} . $$ You need to be doing the latter, not the former, so that you work in the computational basis, $$ \left\{ |00\rangle = |0\rangle\otimes|0\rangle, |01\rangle = |0\rangle\otimes|1\rangle, |10\rangle = |1\rangle\otimes|0\rangle, |11\rangle = |1\rangle\otimes|1\rangle \right\}. $$ That means, in particular, that the state $|\psi\rangle\otimes |\phi\rangle = |0\rangle \otimes |1\rangle = |01\rangle$ gets represented by the vector $$ \begin{pmatrix} 1\begin{pmatrix}0\\1\end{pmatrix}\\ 0\begin{pmatrix}0\\1\end{pmatrix} \end{pmatrix} = \begin{pmatrix} 0 \\ 1 \\ 0 \\ 0 \end{pmatrix}, $$ which remains invariant under the CNOT matrix you've (correctly) given.
More generally, you seem to be having trouble interpreting vectors in a column format. So, to be clear: the amplitudes in each entry correspond to the probability amplitude that the system is in the corresponding basis state, so that e.g. the state $$ \begin{pmatrix} \alpha \\ \beta \\ \gamma \\ \delta \end{pmatrix} \leftrightarrow \alpha |00\rangle + \beta |01\rangle + \gamma |10\rangle + \delta |11\rangle $$ has a probability $|\beta|^2$ to be in the state $|00\rangle = |0\rangle\otimes|0\rangle$$|01\rangle = |0\rangle\otimes|1\rangle$, and so on.