It is well known that there don't appear to be magnetic poles. In Maxwell's equations this has the implication $$ \nabla \cdot \mathbf{B} = 0 $$ and results in the statement "the magnetic field forms closed loops".
But how does this imply closed loops?
After all, in a region where there are no charges you have $$ \nabla \cdot \mathbf{E} = 0 $$ but this wouldn't lead you to say that "the electric field forms closed loops".
Let us begin with the Maxwell's equations for electrostatics and magnetostatics.
For electrostatics: $$\nabla\cdot \mathbf{E}=\frac{\rho}{\epsilon_0} \\ \nabla\times \mathbf{E}=0 $$
For magnetostatics: $$\nabla\cdot \mathbf{B}=0 \\ \nabla\times \mathbf{B}=\frac{\mathbf{J}}{c^2\epsilon_0}$$
The equation $\nabla\cdot \mathbf{B}=0$ in magnetostatics says that the divergence of the magnetic field is always zero. Comparing this with the analogous equation $\nabla\cdot \mathbf{E}=\dfrac{\rho}{\epsilon_0}$ in electrostatics, we can conclude that there are no magnetic charges. This implies that the magnetic field lines neither starts nor ends (for exceptions, see this paper). Then what is the origin of the magnetic field?
Magnetic fields are always associated with electric currents, and the equation $\nabla\times \mathbf{B}=\dfrac{\mathbf{J}}{c^2\epsilon_0}$ says that the curl is proportional to the electric current density. So the magnetic field lines in seldom cases form closed loops around the current vectors (for the general case, see this post).
Now the question is: Why don't electric field lines form closed loops?
At first sight, one might think that electric field lines also form closed loops when $\rho=0$. But this is not the case. To understand this, look at the equation $\nabla\times \mathbf{E}=0$ which says that the curl of electric field is always zero. This simply means that electric field lines can never form closed loops. This fact is closely related to conservative nature of the electrostatic field.
In the field line picture, $\vec\nabla\cdot \vec B=0$ means that the lines don't end (because at the point where they end the divergence would be nonzero). (This is slightly heuristic, but generally correct.)
In the case of the magnetic field, this statement holds everywhere, so magnetic field lines end nowhere. Closed lines are thus the natural choice.
(Note that teh situation can be more complicated, see e.g. Must magnetic field lines close upon themselves or go to infinity?)
For the electric field, $\vec\nabla\cdot \vec E=0$ only holds in limited charge-free areas, hence the lines can simply enter the area and leave again, without closing onto themselves. (They could of course, subject to Maxwell's other equations, i.e. changing magnetic field, as Richard's answer indicates.)
Note, however, that even magnetic lines don't have to be closed when you take the hint: A constant uniform field $\vec B=(0,0,b)$ corresponds to evenly spaced straight infinite field lines and satisfies Maxwell's equations just fine.
