It just boils down to that:
we don't expect an increasing field.
What we're doing in those kinds of derivations is finding interesting and useful solutions of the Maxwell equations. The derivations up to that point essentially pin down the possible solutions of the equations down to a pair of linearly independent functions,
$$
\mathbf E_+(\mathbf r,t) = \mathrm{Re}\left(\mathbf E_0e^{+\kappa x}e^{i(k_z z-\omega t)}\right)
\quad \text{and} \quad
\mathbf E_-(\mathbf r,t) = \mathrm{Re}\left(\mathbf E_0e^{-\kappa x}e^{i(k_z z-\omega t)}\right).
$$
Those are both valid solutions, but they're not both useful, since one of them (say, $\mathbf E_+$, when taken over the $x>0$ half-space) requires a divergent amount of energy to set up, and that is not descriptive of the kinds of situations we can set up in any real experiment. Since it is not useful to us, we discard it.
If this feels like a non-rigorous way to do things, keep in mind that rigour is of limited use here: we're not trying to characterize all possible solutions of the Maxwell equations under some set of boundary conditions (though we also do that, later), we're just trying to find some subset of solutions which are useful for describing real-world experiments (or, more precisely, which form a useful basis for the finite-energy beams used in real-world experiments).
If you do want to do things rigorously, then yes, there are ways to restrict the problem to just those fields that take a reasonably finite amount of energy to set up, and to implement those as boundary conditions that rule out that kind of solution. That one gets tricky, though, because whenever you have plane waves you have an infinite-energy configuration to begin with, but it can be done.
