My very limited understanding of magnetism is that it is essentially stored energy. However what I'm confused on is about the following situation: If you have a compass and you move a magnet close to it the compass will move. My question is, where is that energy coming from? Is the magnet losing some magnetism or energy related to the magnetism? And if so does that mean that a "permanent" magnet would actually lose magnetism faster if you were to bring it around things that react to magnets easily than if it were left in an area devoid of such things?
When you move a magnet near a compass, you are changing the field to do so. After the compass needle settles into position, moving the magnet away takes a bit of extra force (because of the nearby compass needle). So, ultimately the energy source is your hand moving the compass body, or moving the magnet.
The immediate energy source is the stored magnetic field energy, though: a nearby magnet and an aligned compass needle has less stored energy than a distant magnet and nonaligned compass needle.
Magnetic field energy is the square of the local field times the local volume, summed over all space. A compass needle takes magnetic flux in at the near-the-magnet end and releases it at the far-from-the-magnet end, but that lessens the field adjacent to the needle (in field lines model, the field lines converge into the needle, and that leaves fewer field lines per square meter, i.e. less magnetic field, in the space flanking that needle).
Solving magnetic field energy and force problems is the major theme of motor and generator design.
does that mean that a "permanent" magnet would actually lose magnetism
Permanent magnets are only stable in some odd electron-orbital-overlap materials (ferromagnetism is almost a chemical bond effect). So, only application of more energy can lose the magnetism of such materials (like melting an ice cube). When one demagnetizes a non-permanent magnet, it isn't really (microscopically) nonmagnetic, it's just randomly re-oriented patches. External field can be diminished by that randomness, and the rate at which this occurs is the difference between permanent magnet "hard" ferromagnets, and "soft" ferromagnets.