You're doing a direct sum instead of a tensor product of the two vectors; as such, your input state is wrong.
A direct sum operates by concatenating the two vectors:
$$
\psi \oplus \phi
=
\begin{pmatrix}\psi_1\\\psi_2\end{pmatrix}
\oplus
\begin{pmatrix}\phi_1\\\phi_2\end{pmatrix}
=
\begin{pmatrix}
\begin{pmatrix}\psi_1\\\psi_2\end{pmatrix}\\
\begin{pmatrix}\phi_1\\\phi_2\end{pmatrix}
\end{pmatrix}
=
\begin{pmatrix}\psi_1\\\psi_2\\\phi_1\\\phi_2\end{pmatrix}
.
$$
In a tensor product, on the other hand, you form a 'vector of vectors', whose entries are products of the components of the initial two vectors:
$$
\psi \otimes \phi
=
\begin{pmatrix}\psi_1\\\psi_2\end{pmatrix}
\otimes
\begin{pmatrix}\phi_1\\\phi_2\end{pmatrix}
=
\begin{pmatrix}
\psi_1\begin{pmatrix}\phi_1\\\phi_2\end{pmatrix}\\
\psi_2\begin{pmatrix}\phi_1\\\phi_2\end{pmatrix}
\end{pmatrix}
=
\begin{pmatrix}\psi_1\phi_1\\\psi_1\phi_2\\\psi_2\phi_1\\\psi_2\phi_2\end{pmatrix}
.
$$
You need to be doing the latter, not the former, so that you work in the computational basis,
$$
\left\{
|00\rangle = |0\rangle\otimes|0\rangle,
|01\rangle = |0\rangle\otimes|1\rangle,
|10\rangle = |1\rangle\otimes|0\rangle,
|11\rangle = |1\rangle\otimes|1\rangle
\right\}.
$$
That means, in particular, that the state $|\psi\rangle\otimes |\phi\rangle = |0\rangle \otimes |1\rangle = |01\rangle$ gets represented by the vector
$$
\begin{pmatrix}
1\begin{pmatrix}0\\1\end{pmatrix}\\
0\begin{pmatrix}0\\1\end{pmatrix}
\end{pmatrix}
=
\begin{pmatrix}
0 \\ 1 \\ 0 \\ 0
\end{pmatrix},
$$
which remains invariant under the CNOT matrix you've (correctly) given.
More generally, you seem to be having trouble interpreting vectors in a column format. So, to be clear: the amplitudes in each entry correspond to the probability amplitude that the system is in the corresponding basis state, so that e.g. the state
$$
\begin{pmatrix}
\alpha \\ \beta \\ \gamma \\ \delta
\end{pmatrix}
\leftrightarrow
\alpha |00\rangle + \beta |01\rangle + \gamma |10\rangle + \delta |11\rangle
$$
has a probability $|\beta|^2$ to be in the state $|01\rangle = |0\rangle\otimes|1\rangle$, and so on.