They rotate because they are produced by matter that has net angular momentum, and angular momentum is conserved is axially symmetric space-time. So, there's nothing unusual making them rotate that's different from any other physics.
However, you are absolutely right to object that rotation of an infinitesimally small point wouldn't make much sense. In quantum mechanics, we talk about infinitesimally small particles having intrinsic angular momentum ("spin") but this is a uniquely quantum effect and General Relativity is a classical theory. So, your question is a good one. Fortunately, it has a simple answer: the singularity of a rotating black hole in GR is not a point, it's a ring around the black hole's axis of rotation. A rotating ring - even an infinitesimally small one - is sensible because it's topologically distinct from a zero dimensional point.
As a side note, physicists don't tend to believe singularities are real. The general feeling is that quantum gravity will turn them into something more physical. However, it is at least reassuring that GR still makes sense regardless.