I'm actually not sure if this is the $\mathbb{Z}_2$ lattice gauge theory (LGT). This is because $\mathbb{Z}_2$ LGT needs to have spins on top of the edges of the square lattice and not the vertices. These give you different models. The way it's explained in the original question
\begin{equation}
\sigma^z(\vec{x},\vec{\mu})\sigma^z(\vec{x}+\vec{\mu},\vec{\nu})
\sigma^z(\vec{x}+\vec{\mu}+\vec{\nu},-\mu)
\sigma^z(\vec{x}+\vec{\nu},-\vec{\nu})
\end{equation}
contains at least one mistake ($\mu$ without the $\vec{}$ sign), and seems a bit off. If we are using the usual square lattice and the lattice points can be written as $(x,y)$ with $x,y\in\mathbb{Z}$, then something like
\begin{equation}
\sigma^z(x,y)\sigma^z(x+1,y)
\sigma^z(x,y+1)
\sigma^z(x+1,y+1)
\end{equation}
would totally make sense. This corresponds to having spins on top of the vertices of the square lattice and for each plaquette you have a four-body term.
The $\mathbb{Z}_2$ LGT on the other hand, will have terms that look like
\begin{equation}
\sigma^z(x+1/2,y)\sigma^z(x+1/2,y+1)
\sigma^z(x,y+1/2)
\sigma^z(x+1,y+1/2)
\end{equation}
where the $+1/2$ corresponds to having spins in the middle of the edges instead of vertices.
If you were actually interested in the former, that is called the plaquette Ising model (in $2d$), and it has subsystem symmetry, which is different from the gauge symmetry $\mathbb{Z}_2$ LGT has. A quick demonstration of this fact is that if you flip 4 spins on the four edges coming out of a single vertex in LGT, that will give you exactly the same energy. This is exactly what local/gauge symmetry is in this model, and if you have $N$ vertices, you have $O(2^N)$ different states that give you exactly the same energy. This is like the $\mathbb{Z}_2$ symmetry the original Ising model has, but in steroids (there you only had 2 in general). On the other hand, the plaquette Ising model doesn't have that, but you can still flip $L=\sqrt{N}$ spins that are on a single $x$ axis. This gives you $O(2^{2L})$ degeneracy, which is intuitively something between global symmetry and local symmetry, named subsystem symmetry. You can search for papers that talk about plaquette Ising model and they show first-order transitions, but you need to define different order parameters.
Judging from your picture, I'm actually thinking that maybe you were considering the LGT, and not PIM. In that case, I guess I just got confused from the way you wrote the Hamiltonian.