Are self-consistency arguments logical/rigorous? An example with Pauli paramagnetism I am never yet satisfied by arguments in physics that go along the lines of 'assume A is true', 'one way of making A true is for B to happen', 'we check that if B happens then A is indeed true therefore B must happen'. I always think, what if another way of making A true is for C to happen? We have never ruled this out.
To make this explicit, in this book the author calculates the change in spins of a free electron gas due to an applied magnetic field by - assuming the chemical potential will not change, stating one physical process where this is possible, then showing that the chemical potential has indeed not changed and therefore arguing that the physical process is true.

I actually think (from other sources) that in this case $\mu$ actually may not be constant (but is to a certain order of $B$), but that is beside the point - my main question is (even if $\mu$ is constant) how can we proceed to do more physics with the alleged 'result' that $\frac{g(E_F)}{2} \mu_B B$ spins flip when maybe there's some other setup of spin flips that also keep $\mu$ constant instead?
 A: The gist of such self-consistent is the (implied) assumption of a unique solution. If we indeed have a unique solution then the self-consistent solution we find under certain assumptions is indeed the correct one.
In the case you discuss here it is indeed the case, and you can show this by writing the equation for the total number of particles as a function of chemical potential: it is monotonously increasing. This is not, however, guaranteed should be evaluated in a case-by-case basis. A simple example in which we have no uniqueness is in the mean-field Ising model, in which one has to solve the self-consistent equation
$$m=\tanh \beta m$$
The solution $m=0$ is always a solution, but for $\beta>1$ we get a second one. To choose which one of them is the physical one you must resort to a different argument (in that case, which minimizes the free energy.
Many times proving uniqueness is simply too hard and we just have to say that the solution we found is the correct one. There are known cases in which a self-consistent solution is believed for many years to be correct, but it is still unproven, and in some cases it is also proven wrong. A favorite example of mine is in the Sherrington-Kirkpatrick model of spin-glasses. There a solution to a (very difficult) self-consistent equation was found, but yielded unphysical results. Only a few years later a different solution (with an additional symmetry breaking) was proposed and was shown to be favored. This is now the accepted solution, but as much as I know it's rigorous validity is not proven.
A: Just citing your comment. You say that the statements $A$, $B$ and $C$ you criticize is

A = chemical potential is unchanged, B = this specific rearrangement of spins being flipped, C = any other rearrangement of spins those net effect is to keep the chemical potential unchanged

If I understand you correctly, you want to know why the reasoning

*

*assume $A$

*show that $A$ implies $B$
should preclude $C$, which is also consistent with $A$. You can prove this from logic. If the rearrangement of spins that is part of $C$ is not the one in $B$, then
$$ C \Rightarrow \neg B. \tag{$\Delta$}$$
On the other hand, we just showed that
$$ A \Rightarrow B,$$
which can be transposed to
$$ \neg B \Rightarrow \neg A.$$
These three implications show that
$$ C \Rightarrow \neg B \Rightarrow \neg A,$$
so if $C$ were true (and, what we assumed, inconsistent with $B$), $A$ couldn't be true. This chain can also be inverted,
$$ A \Rightarrow B \Rightarrow \neg C,$$
where the final implication is just the transpose of ($\Delta$).
So from a logical stand point, if you assume $A$ and show that it implies $B$, there is no possibility for anything that is not consistent with $B$ to be compatible with $A$. If $A$ were not true, i.e. if the chemical potential should change, then of course the stage is open to any other porposition $C$.
This mostly repeats the last paragraph of your question, but just for summary: Often, one starts from a simple assumption like the one that the chemical potential doesn't change, which of course is true for the field $B = 0$ (sorry for the double use of $B$). So by assuming that there are no non-analyticities, (which may be a dangerous assumption in a $T = 0$ model) you can start from some reference case and add perturbations to it that lead you from the simplified picture to the more realistic one. However, often it is not that bad to have a simplified idea of what happens.
