I am writing a new answer so that psx community can judge my idea regarding the energy issue independently.
The work done in order to assemble the pieces together will simply be converted to the magnetic potential energy of the system. When the pieces are put together, the system will have a tendency to blow apart without involving any singularity and demagnetization is unnecessary(the magnetic field outside the "meat" will decreases as more pieces are put together). Even though the magnetic field vanishes everywhere except in the "meat", it does not mean that there is no magnetic potential energy.
The total magnetic field in all space says nothing about the amount of extractable potential energy of the system.
Mathematically the total magnetic field energy of a system is
$E=\frac{μ_0}{2} \int B_{tot}^2 dV$
Where $B_{tot}=Σ B_i$ , $B_i$ is the magnetic field due to a small portion of magnet.
However not all of this energy is extractable, the extractable potential energy of the system is the negative* of sum of the cross terms
$U=-\frac{μ_0}{2} \int (Σ_{i≠j} B_i B_j) dV$
We know that it is mathematically possible to have $B_{tot}=0$ everywhere while having $Σ_{i≠j} B_i B_j\neq0$. So it is possible that $U\neq0$ in such case, which removes the paradox.
We can also write the magnetic potential energy in terms of magnetic moments
$U=-∫\vec{B_{ext}} . \vec{dm}$
Where $\vec {dm}$ is infinitesimal magnetic dipole element, and $\vec{B_{ext}}$ is the total magnetic field at the dipole excluding the magnetic field due the dipole itself.
Though we assume that the magnetization is permanent, meaning that $\vec{dm}$ is unchanged, there is nothing that prevents $\vec{B_{ext}}$ from changing. If we draw the field lines, we can see that it is more likely that $\vec{B_{ext}}$ on a single piece of magnetic dipole increases as more and more segments are put together. We can also see that $\vec{B_{ext}}$ tends to be anti-parallel to $\vec {dm}$. Therefore the total potential energy will be positive and its value increases as more and more pieces are put together. Which suggests that the segments tend to blow apart when put together. Due to the geometry of the system it is hard to give a rigorous proofs of my last few statements. However, no matter how complicated the system is, we can still be sure that there are many rooms for the energy to hide. Thus the conservation of energy is still safe.
*Note that the negative sign here is not obvious. In the case of electric charges we would pick the positive of the sum, instead of its negative. The explanation of the sign can be found in my blog http://wp.me/p1AI8o-2P; It is also explained briefly in Feynman Lectures on Physics Vol.2 ch.15.