As mentioned in another answer, it is incorrect that ${\bf E}$ and ${\bf B}$ are "oscillating along the direction of ${\bf k}$".
Blackbody radiation in a conducting cube is a nice way to see that the electric field must always point perpendicular to the wave vector. If you solve the electric field wave equation inside of a conducting cube of length $L$, you find that the electric field has components:
$$
E_x({\bf r}, t) = E_{0x} \, \sin(\omega_{\bf n} \,t) \, \cos \left( \frac{n_x \pi x}{L} \right) \, \sin \left( \frac{n_x \pi y}{L} \right) \, \sin \left( \frac{n_z \pi z}{L} \right)
$$
and similar expressions for $E_y$ and $E_z$, where ${\bf n}$ is a vector of non-negative integers. We get a countable set of solutions, labeled by ${\bf n}$, because of the finite box. The particular dependence on the $\cos$ and $\sin$ functions ensure that, at the conducting boundary, the component of the electric field parallel to that boundary vanishes (so that there is no moving charge in the conductor). An arbitrary electric field solution is found as a superposition of these ${\bf E}^{({\bf n})}$ solutions.
But there is one more constraint from Maxwell's equations that has not yet been applied. It turns out that the direction of the electric field vector, ${\bf E}_0 = \left( E_{0x}, E_{0y}, E_{0z} \right)$ is not allowed to be completely arbitrary. Applying Gauss's Law (in free space) to the above solution yields:
$$
\nabla \cdot {\bf E} = 0 \quad \rightarrow \quad E_{0x} n_x + E_{0y} n_y + E_{0z} n_z = 0 \quad \rightarrow \quad {\bf E}_0 \cdot {\bf n} = 0
$$
Thus we see that for a particular solution labeled by ${\bf n}$, the electric field vector is always perpendicular to ${\bf n}$. It turns out that ${\bf n}$ is also the direction of propagation: it is the direction you get from ${\bf E} \times {\bf B}$.
Applying a similar analysis to a cubic mass-and-springs model of a solid yields almost the same solutions (because we are again seeking solutions to a wave equation). But in that case there exists no physical law that displacements be divergence-free, so longitudinal oscillations are allowed.
So, both EM waves and mechanical waves are determined by solving a wave equation. But EM waves must also satisfy the other parts of Maxwell's equations, namely Gauss's Law and $\nabla \cdot {\bf B} = 0$. These prohibit longitudinal oscillations of the the electric and magnetic fields.
