When drawing Feynman diagrams, it is important to fix the incoming and outgoing particles and their momenta. For the inexperienced, this is ideally done before drawing the rest of the diagram in order to avoid confusions like yours. So, let's assign each state some momentum: Let's give the electron in the upper left corner (4-)momentum $p_1$, lower left $p_2$, upper right $p_3$ and lower right $p_4$. Let us consider all momenta as 'flowing' from left to right (this is an arbitrary choice that makes no difference to the actual calculations).
Then, it immediately becomes clear why the two diagrams are different. For instance, check out the momentum on the photon line (call it $q$, and imagine it flowing top to bottom) that connects both vertices. Conservation of (4-)momentum forces always applies, so we can write it in terms of the momenta flowing into the vertices. For the $t$-channel diagram, it is clear that $q=p_1-p_3=p_2-p_4\equiv \sqrt{t}$. However, for the $u$-channel diagram $q=p_1-p_4=p_2-p_3\equiv \sqrt u$. Thus, the photon carries a different momentum. When carrying through the calculation, it will be seen that this makes a serious difference.
There is a more general point to be made here: Feynman diagrams can be very misleading, especially to those that are not (all too) familiar with the actual calculations that they visually represent. When it really comes down to it, it is of vital importance to realize that the diagrams should always be considered as secondary aids in performing calculations, which can just as well be done without drawing anything (it just takes a lot longer): Always be very careful when interpreting the diagrams.
