It helps to remember that ferrofluid is simply a suspension of iron-containing particles. The shape you see is the iron particles attempting to follow the magnetic field lines, just like iron filings around a magnet (in 3D, rather than on a surface as is commonly demonstrated). The shape is not exactly the same as iron filings around a magnet, because of the surface tension of the liquid.
In a bit more detail: magnetic force on suspended particles throughout the material leads to a net force at the surface, where the magnetic permeability drops off. This causes a force normal to the surface, which results in a protrusion of fluid. As the protrusion grows, the surface is stretched, increasing surface tension force until it is strong enough to balance out the magnetic normal force. The equilibrium shape is therefore a peak, which is taller and thinner when the field is stronger.
Why do these peaks form in some parts of the fluid, while others are left as valleys?
The places where peaks form are essentially random, nucleated by any small bump in the fluid surface, which will then grow. That is why they are called "instabilities", since a perfectly stable surface would not have any preferential position to form a peak (though there are no perfectly stable surfaces in reality).