The answer by Pieter is a start, but it’s incomplete. Magnetization is a local average of the spins, and it can vary from point to point. In larger minerals, it is organized into regions called domains. Inside each domain, the magnetization can be in one of 2-8 possible directions (easy directions) that are determined by anisotropy in the mineral’s magnetic properties. In minerals below about 0.1 microns in diameter, there is only one domain. If an applied field is not aligned with an easy axis, the magnetization rotates towards it; but when the field is removed, the magnetization rotates back to an easy direction. In general, this is not in the direction that the field was in. Only the sum of all the magnetic moments is in the direction of the field.
Without the anisotropy, there is no magnetization in zero field (what paleomagnetists call remanence). However, even with anisotropy, thermal fluctuations in the magnetization can lead to jumps between remanent states. Just below the Curie temperature, jumps occur too frequently to allow a remanence to establish itself, and the minerals are superparamagnetic (like paramagnetic, but with electron spins replaced by moments of entire crystals). The only thing stopping them from jumping from one state to another is the energy barrier in between. This is determined by the anisotropy and the size of the particle. As the temperature decreases to room temperature, there may come a point where the barrier is too large for jumps to occur at an appreciable rate, and the remanence is blocked. The blocking temperature is different for each mineral, and the smallest minerals remain superparamagnetic at room temperature. Because of the thermal fluctuations above the blocking temperature, there is some probability of the magnetization ending up in any one of the easy directions. Thus, this kind of remanence (known as thermoremanent magnetization) is a statistical average of magnetizations that can be in all directions. That is what is really meant by "aligned with the magnetic field".
There is a nice textbook on paleomagnetism that can be legally downloaded for free. See chapters 2 and 3 for more information on how rocks can get magnetized.