As Anode's answer shows, the surface brightness is proportional to the square of the angular diameter distance, for the same amount of total flux. And the angular diameter distance is equal to luminosity distance divided by $(1+z)^2$. ($D_A = D_L/(1+z)^2$. This is Etherington's reciprocity theorem.) So when you relate the surface brightness to the luminosity distance, you square the factor of $(1+z)^2$ to become a factor of $(1+z)^4$. So your question basically boils down to why the angular diameter distance itself is equal to luminosity distance divided by $(1+z)^2$.
I am not familiar with the derivation of Etherington's reciprocity theorem and the following is my layman's superficial interpretation. If you look at Distance measure#Overview, you see that $D_A = D_M/(1+z)$, and $D_L = (1+z)D_M$, where $D_M$ is the transverse comoving distance (which is equal to $D_C$, the comoving distance, in a flat universe; for simplicity, we will treat $D_M$ as the comoving distance since our universe is practically flat). So the $(1+z)^2$ factor can be broken down into two parts: the $(1+z)$ factor between $D_A$ and $D_M$, and the $(1+z)$ factor between $D_M$ and $D_L$.
When we look at a faraway object, the angle that it takes up in the sky is the same angle that it actually took up compared to our location when the light was emitted (even though we couldn't see it back then). The uniform expansion of a flat universe does not change the angles of the light. Since the angular diameter distance is inferred based on the size of the object when the light was emitted, plus the angle in the sky we see now (which is the angle it took up in reality when the light was emitted), the angular diameter distance would basically equal the distance the object was from us when the light was emitted. This distance is a factor of $(1+z)$ smaller than the comoving distance (i.e. the distance where the object would be now if it followed the Hubble flow) simply because the universe (and thus the distance between us and the object) expanded by a factor of $(1+z)$ between when the light was emitted and now. This explains why $D_A = D_M/(1+z)$.
(An equivalent way to think about this is that faraway objects appear to us where it would be located today, if it simply followed the Hubble flow and had no other interactions. If the parts of the object just followed the Hubble flow, then the diameter of the object would have expanded by a factor of $(1+z)$ from when the light was emitted until now, and it appears to us to be at the location it would be if it had expanded in this manner. (Of course, the object is probably bound gravitationally and thus didn't expand as the universe expanded, but the light doesn't know that.) Thus, we see the object as if it were $(1+z)$ times the size it would be today if it had stayed the same size, and thus our calculation of the distance based on the angular diameter and the size of the object will be too small by a factor of $(1+z)$ compared to where it is currently located today.)
The light that the object emitted when it was emitted is now spread over a sphere with radius equal to the comoving distance (the distance between us and the object today). So at first, you would expect the luminosity distance to be equal to the comoving distance, and that would be true if we were not moving relative to the object. However, since we are moving away from the object, two additional factors come into play.
The first is that events in the object play out in "slow motion" for us. This happens whenever there is a Doppler effect -- a song played moving towards you will sound sped up, and a song played moving away from you will sound slowed down. The redshift is a direct measure of the amount of "slow motion", as redshift is just the wiggling of the electromagnetic fields playing out in slow motion (and thus lower frequency). This means that the rate of photons reaching us will go down by a factor of $(1+z)$ -- if it would have been 1 photon per second for an observer not moving with respect to the object, it would now be 1 photon per $(1+z)$ seconds for us.
The other effect is that each photon of light has less energy, due to the redshift. The frequency decreased by a factor of $(1+z)$, and thus the energy of each photon decreased by a factor of $(1+z)$. These two factors combine to decrease the flux by a factor of $(1+z)^2$, and since the luminosity distance is proportional to the inverse of the square root of the flux, a decrease in flux by a factor of $(1+z)^2$ increases the luminosity distance by a factor of $(1+z)$. This explains why $D_L = (1+z)D_M$.