This isn't a fundamental property of a permanent magnet: it depends on details of the materials used in its manufacture, on the process by which it was magnetized, by its history of exposure to other magnetic fields, and a bunch of other "weather." People who need uniform fields build Helmholtz coils.
It's probably possible to build a magnet with a uniform magnetization over the surface. It's definitely possible to build a magnet where the magnetization is not uniform. And it's possible to take a uniformly magnetized hunk of stuff and screw it up.
I used to teach a poorly-designed lab experiment with solenoids and a bunch of old AlNiCo magnets, where the students had a 10% chance of accidentally reversing the polarity of their bar magnet with the solenoid. By the end of the week, when all dozen intro lab sections had learned on the same equipment, I would usually spend an afternoon re- polarizing the magnets so that they matched their labeling, and I would usually discover one magnet that some students had converted into a quadrupole! Switching to the newer rare-earth magnets reduced the chance of an accidental mispolarization, but didn't eliminate it.
That's before we start to talk about the "patch effect" in polycrystalline metals.
The patch effect is usually important in precision electrostatics experiments (for example): parts of a metallic surface which present different crystal planes have different work functions, so there is some non-removable variation in the electrical potential across the surface. I haven't encountered a discussion of the patch effect in surface magnetism, but the phenomenon of magnetic domains is clearly closely-related.
As a rule of thumb, if you want your magnetic field to be uniform over some length scale $\Delta x$, then the distance between your should-be-uniform field region and you magnetic field source should be several times larger than $\Delta x$. There is a nice mathematical technique for estimating just how much larger.