Block diagonalizing a spin-chain Hamiltonian $\newcommand{\ket}[1]{\left|#1\right>}$
I am learning about exact diagonalization methods, currently following this explanation. My question is in regards to the part where we utilize the fact that the Hamiltonian conserves the total spin to block-diagonalize it. In this explanation it says that "block-diagonalization just amounts to including states with only given $m_z$," and the process is to simply compute individual blocks with different total magnetization. 
My question is, how is this different than simply rearranging the order of the states? If I initially construct the Hamiltonian in the "binary basis", where $\ket{0} \equiv \ket{\cdots 0000} \equiv \ket{\cdots \downarrow\downarrow\downarrow\downarrow}, \ldots, \ket{5} \equiv \ket{\cdots 0101} \equiv \ket{\cdots \downarrow \uparrow \downarrow\uparrow}$, etc, then I want to reorder the basis such that the states are grouped by their total spin $m_z$, and I do this by permuting the rows and columns of the Hamiltonian which I obtained in the binary basis. 
For example, for $N=4$, the states $\ket{3}\equiv \ket{0011}$ and $\ket{5}\equiv \ket{0101}$ both have $m_z = 2$, while $\ket{4} \equiv \ket{0100}$ has $m_z = 1$. So I swap rows 3 and 4 and then columns 3 and 4, which I thought would just correspond to reordering the basis states as $\left\{\ket{0},\ket{1} ,\ket{2}, \ket{4},\ket{3},\ket{5},\ldots\right\}$. It seems to me that if I reorder the states such that all those with the same $m_z$ are adjacent then the resulting matrix should be block diagonal.
I have written a code to swap rows and columns of the Hamiltonian such that they are ordered from maximum $m_z$ to minimum $m_z$, but the resulting matrix is not block diagonal. So do I have a bug in my code or is my logic incorrect? 
 A: [Note: I'm not happy with the values you assign to overall magnetization, for me, it's more natural to think of, say, $|\downarrow \uparrow \downarrow\uparrow\rangle$ as having $m_z=0$ (ups and downs cancel each other), and I'll use this terminology in my answer.]
This might only be a partial answer to your question, but it should help you moving forward. The central problem, I believe, is that you're swapping the wrong rows/columns.
Consider the case $N=3$ of the spin-$\frac{1}{2}$ $XXX$ model, which is the first non-trivial example (in the sense of not already being ordered by total magnetization). In the bit notation you introduced, you'll find that $|1\rangle$, $|2\rangle$ and $|4\rangle$ have $m_z=-1$, while $|3\rangle$, $|5\rangle$ and $|6\rangle$ have $m_z=1$. To obtain a block-diagonal Hamiltonian in spin-$\frac{1}{2}$ representation (i.e. local spin operators are represented by the familiar Pauli matrices), you would want to swap those parts in the Hamiltonian $H$ that affect the states $|3\rangle$ and $|4\rangle$.
By representing a spin up through $|\uparrow\rangle=(1,0)^T$ and using the Kronecker product to express $|3\rangle=|\uparrow \uparrow \downarrow\rangle=|\uparrow\rangle \otimes |\uparrow\rangle\otimes | \downarrow\rangle$ etc, you'll find that $|3\rangle$ is an eight-dimensional vector which has a "1" in position 2, while $|4\rangle$ is a vector with zeroes everywhere apart from position 7. Clearly, the 2nd (resp. 7th) column of the matrix $H$. Ideally, write down the 3-site matrix explicitly to check if that does the job.
I also recommend this introduction to spin chains which might clear up some confusion.
A: Simply rearranging the states according to $m_z$ DOES make the Hamiltonian block-diagonal. How did you check that the resultant is block-diagonal? Maybe you generated the block-diagonal Hamiltonian the way Sandvik suggests and then compared it your rearrangement. The problem here is that Sandvik arranges states in ascending order even within each block. I added an additional term $\dfrac{i}{2^{N+1}}$ to the $m_z$ value of each state represented by integer $i$ and then sorted them which introduces correct order within blocks and compared that to Sandvik's approach by explicitly constructing each block and then making a block-diagonal matrix using $\mathtt{scipy.linalg.block\_diag}$.
