I have been trying my very best to stay out of this question because @Bence Racskó's answer is already very good, but alas, it nags at me. Now I will be wasting a lot of time just to make a small improvement upon his answer.
Let us work backwards. The generalised Stokes's theorem that encompasses all three as special cases, is
$$
\int_\sigma \mathrm d \omega = \int_{\partial\sigma} \omega
$$
if $\sigma = \Gamma$ is a 1D curve starting @ $\vec r _1$ and ending @ $\vec r _2$, and $\omega$ is the zero-form, scalar function $\phi$, then we get
$$
\int_\Gamma \mathrm d \phi = \int_{\partial\Gamma} \phi\\
\int_\Gamma \vec{\mathrm d r} \cdot \vec{\nabla\phi}
= \phi ( \vec r _2 ) - \phi ( \vec r _1 ) \tag{Fundamental Thm of Calculus}
$$
if $\sigma = \Sigma$ is a 2D surface with boundary loop $\partial\Sigma$, and $\omega = \vec A \cdot \vec{\mathrm d r}$ is a one-form,
$$\tag{Stokes's Theorem}
\int_\Sigma \vec{\mathrm d S} \cdot \vec\nabla \times \vec A
= \int_{\partial\Sigma} \vec{\mathrm d r} \cdot \vec A
$$
and finally, if $\sigma = V$ is a 3D volume with boundary surface $\partial V$, and $\omega = \vec{\mathrm d S} \cdot \vec B$ is a two-form, then
$$\tag{Gauß's Law}
\int_V \mathrm d V\ \vec\nabla \cdot \vec B = \int_{\partial V} \vec B \cdot \vec{\mathrm d S}
$$
Now, if you accept Bence's answer, which boils down to ``There is no boundary of a boundary", which is, in itself, a rather geometric concept written in rigorous maths, then you have a stronger answer than what I am about to present, because I will be assuming nice and smooth stuff for the sake of clarity. I just felt that it is somewhat sad to reduce a beautiful geometric concept to abstract manipulation.
Let us now look at the geometric understanding of the grad. The stereotypical understanding of
$\phi$ is the electrostatic potential, which, in turn, was a borrowed concept from gravitational potential. This is a function, everywhere in space, that gives a height. You can visualise smooth hills and valleys of any complexity. The nice part of being able to do this is that you can leverage that you know how hills and valleys are like, and maybe you might know about contour lines. The gradient
$\vec{\nabla\phi}$ is a vector field that always points towards the hill-tops and away from valleys, in the direction of steepest ascent, and thus must always be perpendicular to contour lines.
Now let us prove that curl grad vanishes. That means we substitute $\vec A = \vec{\nabla\phi}$ into Stokes's Theorem and try to define things by considering small neighourhoods of every point in space. That is, we pick a very small area patch $\Sigma$, which we must first insist not to be malformed, i.e. should be reasonably circular or squarish, and use the average of that integral to define its value at that point. We will never actually use the surface integral, instead using the boundary loop integral to define the value of the integral. Now, if we happened to have picked a hilltop or valley bottom, then obviously we could have chosen the necessarily existing infinitesimal contour line surrounding the hilltop or valley bottom, and obviously we get a zero because of the perpendicularity of the gradient and the contour lines as mentioned above. If we do not so happen to pick those special spots, then we can break the boundary loop integral into two parts, and that then allows us to apply the Fundamental Theorem of Calculus to the two parts individually. In general, each of the two parts has a higher and lower end points. One part goes on a path from the lower end point to the higher end point, and the other part is a different path from the higher end point to the lower end point. The important thing is that such a sum necessarily cancels because they are the same end points, cancelling in pairs. Thus, the boundary loop integral of the gradient of any scalar function is necessarily zero, which means that the surface integral about any neighbourhood of any point of a curl of a gradient is zero too. This then implies that the curl of a gradient is zero. i.e. $\vec\nabla\times\vec{\nabla\phi}=\vec0$
Continuing on, we should now discard the link between the vector field $\vec A$ from the gradient of any function. Let us study what the curl of a general vector field is. The three components of $\vec\nabla\times\vec A$ at any point are the integral over small circles (boundaries of discs) aligned in the $yz,\ zx\ \&\ xy$ planes respectively of the vector field $\vec A$ centred at the point. Note that if $\vec A$ has any constant vector field parts (think of uniform wind) or any convergence divergence (think of the gradient field hilltop above, which would always be perpendicular to contour lines, and contour lines near smooth hills and valleys are always approximately circular), then those contributions give zero in the loop integral. The only parts that gives a contribution is when the vector field itself has any swirliness, precisely why it is called the curl.
Now we try to understand why div curl is zero. For this purpose, it is preferable to consider a small cube centred at any point to be the definition. A cube has six surface area squares, and each square is made of four lines. The curl at the centre of the surface area squares is defined as the loop integral of its four lines. The divegence of a general vector field is the surface integral of the top minus the bottom, for some convention, right minus the left, and front minus the back, added together. The divergence of the curl is thus some sum and difference of the loop integrals around the edges of the cube. Everything here is linear, so we can superpose solutions. Let us consider just assigning vectors for the top and bottom squares, leaving the vertical lines field-less. If the top and bottom squares are getting integrals that rotate around in the same direction, then the top and bottom integrals cancel, and the sides integral also cancel out, so that is fine. If the top and bottom are rotating in the opposite direction, then although the top and bottom integrals reinforce, the sides integrals will get just enough contributions to cancel those. Hence, any superposition of the "same rotation direction" and "opposing rotation direction", will cancel out. That allows us to have any distribution of vector fields on the top and bottom edges as we like. Since we may also superpose the side faces too, this means that any general distribution of vectors along the edges of the cube will always get a zero integral. This proves that
$\vec\nabla \cdot \vec\nabla \times \vec A = 0$
Of course, this complicated argument is far less easy to understand than just seeing that the spherical surface integral of the curl = two hemispherical surface integrals of the curl, and that turns into the criss-crossing equatorial integrals, cancelling out. But this is really just the ``there is no boundary of a boundary" argument, which I had initially rejected as being too abstract.
After all that work, I wanted to point out that we have a notation nightmare. Of course, notation nightmares are really impossible to avoid because physicists had just been inventing new maths before mathematicians could have had the time to properly study what those new tools should be unified as.
If you know some tensors and differential geometry (especially forms), then you will know from tensors that we have such a thing as contravariance and covariance. We have been very severely abusing the notation, treating non-vectorial quantities as vectors, causing a lot of confusion. So here I will quickly cover a few facts.
Scalar functions are zero-forms. The gradient is most naturally a one-form. When we say that it "points in the direction of steepest ascent", it really cannot point anywhere because it is still a one-form. That is, in $\mathrm d \phi = \mathrm d x^a \nabla_a \phi$, the $\nabla_a \phi$ is a covariant quantity that is really more like layer cake, and cannot be pointing anywhere. Instead, we must use the metric to raise the index into a contravariant quantity, and then we can identify it with vectors, for it to point anywhere. i.e. $g^{ab} \nabla_b \phi$ is finally a vector that can point towards the direction of steepest ascent.
This extends to momentum $p$ too. Because in quantum theory we have $p_a x^a$, where position coördinates $x^a$ are contravariant by definition, the momenta must be covariant. Or better, the Noether's theorem conserves the covariant components. If you want to see where the momentum points to, then, again, you use the metric to convert it to a vector, but the contravariant version of the momentum have components that are NOT conserved. To make life incredibly sickening, the universally accepted convention is such that contravariant momentum is all positive signs, which means that covariant momentum inherits a negative sign, even though it is the one that is natural. Same with the vector potential.
The electrodynamics vector potential is actually naturally a one-form, which goes well with the momentum in both classical Hamiltonian mechanics and quantum theory. Thus, the Faraday tensor is $$F = \mathrm d A = \mathrm d x^a \wedge \mathrm d x^b F_{ab} = \mathrm d x^a \wedge \mathrm d x^b \left ( \partial_a A_b - \partial_b A_a \right )$$
Or maybe its Hodge dual. Since $\mathrm d \mathrm d = 0$, this means $\mathrm d F = 0$ and that is the internal constraint pair of the Maxwell's equations. The other pair is in $\mathrm d \star \mathrm d A = J$, where $J$ is the current three-form, just ripe for integration. Note that it has the correct units---charge density is just ripe for volume integration. If you want to find the arrow direction of this current density, you have to take the Hodge dual, $\star J$, and then raise the indices to contravariant.
That is, if you want to insist to map 1-forms and vector fields, yes, you have a one-to-one mapping. But then the E and B fields of Maxwell's electrodynamics, is really 2-forms in Minkowski spacetime, and they really are not vectors at all. Considering the curl, which, as seen in the small circles earlier, are really $yz,\ zx\ \&\ xy$ bivectors, if they are even to be sent back as contravariant quantities to look at. This is the source of why we have that silly polar v.s. axial vector nonsense. If you leave them as they are naturally, as 2-forms or as bivectors, then you will not have this problem. Note that forces, like momentum, is naturally a 1-form because for a charged particle moving with velocity $v^b$, the Lorentz force law reads as $F_a = \frac{\mathrm d\ }{\mathrm d t} p_a \propto F_{ab} v^b = F_{ab} g^{bc} p_c / E$
For completeness, the basis vectors of contravariant stuff is $\partial_a$, and for zero-forms, $\partial_a = \nabla_a$. For example, velocity vector acting on potential $\phi$ is $v^a \nabla_a \phi$.
These are the correct stuff, but I have neither the time nor space to explain why. I hope they can still help.