Making sense of the canonical anti-commutation relations for Dirac spinors When doing scalar QFT one typically imposes the famous 'canonical commutation relations' on the field and canonical momentum: $$[\phi(\vec x),\pi(\vec y)]=i\delta^3 (\vec x-\vec y)$$ at equal times ($x^0=y^0$). It is easy (though tedious) to check that this implies a commutation relation for the creation/annihilation operators
$$[a(\vec k),a^\dagger(\vec k')]=(2\pi)^32\omega\delta^3(\vec k-\vec k') $$
When considering the Dirac (spinor) field, it is usual (see e.g. page 107 of Tong's notes or Peskin & Schroeder's book) to proceed analogously (replacing commutators with anticommutators, of course). We postulate 
\begin{equation}
\{\Psi(\vec x),\Psi^\dagger(\vec y)\}=i\delta^3(\vec x -\vec y) 
\end{equation}
and, from them, derive the usual relations for the creation/annihilation operators. 
I'd always accepted this and believed the calculations presented in the above-mentioned sources, but I suddenly find myself in doubt: Do these relations even make any sense for the Dirac field? Since $\Psi$ is a 4-component spinor, I don't really see how one can possibly make sense out of the above equation: Isn't $\Psi\Psi^\dagger$ a $4\times 4$ matrix, while $\Psi^\dagger\Psi$ is a number?! Do we have to to the computation (spinor-)component by component? If this is the case, then I think I see some difficulties (in the usual computations one needs an identity which depends on the 4-spinors actually being 4-spinors). Are these avoided somehow? A detailed explanation would be much appreciated. 
As a follow-up, consider the following: One usually encounters terms like this in the calculation:
$$u^\dagger \dots a a^\dagger\dots u- u\dots a^\dagger a \dots u^\dagger $$ Even if one accepts that an equation like $\{\Psi,\Psi^\dagger\}$ makes sense, most sources simply 'pull the $u,\ u^\dagger$ out of the commutators' to get (anti)commutators of only the creation/annihilation operators. How is this justified?
EDIT: I have just realized that the correct commutation relation perhaps substitutes $\Psi^\dagger$ with $\bar \Psi$ (this may circumvent any issue that arises in a componentwise calculation). Please feel free to use either in an answer.
 A: One usually starts from the CCR for the creation/annihilation operators and derives from there the commutation rules for the fields.
However, one can start from either (see for example here about this).
Suppose we want then to start from the equal-time anticommutation rules for a Dirac field $\psi_\alpha(x)$:
$$ \tag{1} \{ \psi_\alpha(\textbf{x}), \psi_\beta^\dagger(\textbf{y}) \} = \delta_{\alpha \beta} \delta^3(\textbf{x}-\textbf{y}),$$
where $\psi_\alpha(x)$ has an expansion of the form
$$ \tag{2} \psi_\alpha(x) = \int \frac{d^3 p} {(2\pi)^3 2E_\textbf{p}} \sum_s\left\{ c_s(p) [u_s(p)]_\alpha e^{-ipx} + d_s^\dagger(p) [v_s(p)]_\alpha e^{ipx} \right\}$$
or more concisely
$$ \psi(x) = \int d\tilde{p} \left( c_p u_p e^{-ipx} + d_p^\dagger v_p e^{ipx} \right), $$
and we want to derive the CCR for the creation/annihilation operators:
$$ \tag{3} \{ a_s(p), a_{s'}^\dagger(q) \} = (2\pi)^3 (2 E_p) \delta_{s s'}\delta^3(\textbf{p}-\textbf{q}).$$
To do this, we want to express $a_s(p)$ in terms of $\psi(x)$. We have:
$$ \tag{4} a_s(\textbf{k}) = i \bar{u}_s(\textbf{k}) \int d^3 x \left[ e^{ikx}  \partial_0 \psi(x) - \psi(x) \partial_0 e^{ikx} \right]\\
= i \bar{u}_s(\textbf{k}) \int d^3 x \,\,  e^{ikx}  \overset{\leftrightarrow}{\partial_0} \psi(x) $$
$$ \tag{5} a_s^\dagger (\textbf{k}) = -i \bar{u}_s(\textbf{k}) \int d^3 x \left[ e^{-ikx}  \partial_0 \psi(x) - \psi(x) \partial_0 e^{-ikx} \right] \\
=-i \bar{u}_s(\textbf{k}) \int d^3 x \,\, e^{-ikx}  \overset{\leftrightarrow}{\partial_0} \psi(x) $$
which you can verify by pulling the expansion (2) into (4) and (5).
Note that these hold for any $x_0$ on the RHS.
Now you just have to insert in the anticommutator on the LHS of (3) these expressions and use (1) (I can expand a little on this calculation if you need it).

most sources simply 'pull the $u, u^\dagger$ out of the commutators' to get (anti)commutators of only the creation/annihilation operators. How is this justified?

There is a big difference between a polarization spinor $u$ and a creation/destruction operator $c,c^\dagger$.
For fixed polarization $s$ and momentum $\textbf{p}$, $u_s(\textbf{p})$ is a four-component spinor, meaning that $u_s(\textbf{p})_\alpha \in \mathbb{C}$ for each $\alpha=1,2,3,4$. 
Conversely, for fixed polarization $s$ and momentum $\textbf{p}$, $c_s(\textbf{p})$ is an operator in the Fock space. Not just a number, which makes meaningful wondering about (anti)commutators.
