A CNOT gate flips the target bit when the control bit is set to $|1\rangle$. Thus, defining it by $|c\rangle |t\rangle \rightarrow |c\rangle |t \oplus c\rangle $ makes sense to me.
On the other hand, its matrix representation
$$\begin{pmatrix}1 & 0 & 0 &0 \\ 0 & 1 & 0 &0 \\ 0& 0 & 0&1 \\ 0 & 0 & 1 & 0\end{pmatrix}$$
doesn't seem right, because if I multiply it by a vector
$$\begin{pmatrix}a \\ b \\ c \\ d \end{pmatrix}$$
representing the control bit ($a|0\rangle + b|1\rangle$) and the taget bit ($c|0\rangle + d|1\rangle$), I always get
$$\begin{pmatrix}a \\ b \\ d \\ c \end{pmatrix}$$
which means I am flipping the target bit regardless the value of control bit.
Can someone please explain what is wrong with my understanding here?