$\textbf{EDIT}$ A few further clarifications about the symmetries and field of an infinite solenoid:
- For an arbitrary point inside the solenoid (not necessarily on the axis), our two symmetries alone do not imply the field is axial. Here, you need to invoke Gauss' law, which says the flux of $\mathbf{B}$ through any closed surface is zero. If $\mathbf{B}$ developed a radial component as you strayed from the axis, then it would necessarily have non-zero flux through a finite cylinder concentric with the axis.
- To show the field must vanish outside, again apply Ampere's law to a rectangle with two radial and two axial sides, but this time entirely outside the solenoid. Since no current is enclosed, the circulation vanishes, meaning the two axial sides have opposite $\mathbf{B}$. But now split this rectangle into two with another axial segment. By the same reasoning, $\mathbf{B}$ on the new segment must be opposite to each of the previous two axial segments. But these are already opposite to each other, leaving us with an impossible situation unless $\mathbf{B} = 0$ on all 3 segments.
- There is actually a third type of symmetry here (assuming the coils are tight enough that we can treat each turn as a flat circle, which we already assumed when asserting the other symmetries): the solenoid has reflection symmetry along its axis. You might then say that $\mathbf{B}$ should likewise be symmetric under reflection, but this would imply its axial component must be zero, leaving us with no field at all. So why doesn't this symmetry apply to $\mathbf{B}$? The answer is, it does apply -- it's just that $\mathbf{B}$ is not truly a vector! Some call it a pseudovector, but I find it more illuminating to call it a bivector. If a vector is an oriented line with a magnitude, then a bivector is an oriented plane with a magnitude -- oriented in the sense of "clockwise/counterclockwise as viewed from above", although there are nicer intrinsic ways to define orientation. Anyway, when thought of as a bivector, $\mathbf{B}$ lies in the plane of the circular current loop that makes up the solenoid, and its orientation is the same as that of the current. And then you can see that, when we reflect the solenoid along its axis, we don't change the orientation of $\mathbf{B}$ any more than we do current direction.
This last point is absolutely crucial when it comes to more deeply understanding electromagnetism. As you probably know, electromagnetism was largely the impetus for special relativity; and in relativity, the electric and magnetic fields are unified into a single bivector, with 6 components, one for each basis plane of spacetime. (In general it can no longer be thought of as a single oriented plane, as 4D spacetime allows us to have two planes that are entirely orthogonal to each other.) Bivectors are fascinating, and they are actually part of the greater geometric algebra, which in the case of spacetime is called the spacetime algebra. But I should probably leave it at that, as I've already strayed somewhat from the scope of the question.