Is magnetic field due to current carrying circular coil, zero everywhere except at its axis?
No its not. The cartoon of the magnetic field lines you have provided shows as much. The field in immediate vicinity of the wires is non-zero. So is at large distances away from it. In fact its non zero everywhere.
With the premise
Consider a rectangular loop ABCD,where length AB=CD=∞
"your book" writes
Apart from the side along the axis,the integral $\int\vec{B}⋅d\vec{l}$ along all three sides will be zero since $B=0$
Since the length of $BC$ and $DA$ aren't stated, its not correct to say that field along $AB$ is zero. It is zero for $BC$ and $DA$ though, since they are infinitely far away
and field power-decays with distance. The statement holds only if $AB$ is infinitely far away too, which is not clear.
I am guessing the book is trying to illustrate Ampere's law via a direct but simple calculation on a rectangular loop whose three sides are infinitely far away. Since there are no constraints$^1$ on the geometry of the loop in the law, this loop is as good as any even though its effectively just a single line.
Why exactly
$$\int_ {A}^{B}\vec{B}\cdot \vec{dl} + \int_ {B}^{C}\vec{B}\cdot \vec{dl}+
\int_ {D}^{A}\vec{B}\cdot \vec{dl}$$
is zero?
From a theoretical standpoint, you have already proved this for all outer loops connecting $C$ to $D$ from outside the current carrying loop in eqns. $1-3$, since $\int\vec{B}.\vec{dl}$ on the straight line $CD$ already consumes all of the line integral Ampere's law allows.
what if we don't take AD and BC to be infinite? and take them such that they are finitely greater than R. in that case Ampere's law is still applicable and the equations would still imply line integral to come zero.
This ambiguity in the lengths of $BC$ and $DA$ is what I have alluded to above. But as just stated, the result holds independent of the geometry of the outer part of the loop. To see this explicitly is another matter altogether. Specifically$^2$, the line integral of the field along $AB$ will vanish regardless of its distance from the loop as long as its outside the loop and match $CD$ (in magnitude) as long as its inside the loop.
$^1$ apart from those imposed by mathematical rigour like the loop can't pass through the wire.
$^2$As an instructive example, lets assume $AB$ is at distance $D>R$ from loop center, coplanar and parallel with $CD$ and normal to the plane containing the loop. Using units $R=1,\frac{\mu_0 I N}{4\pi}=1$, and carefully using the symmetry of the following geometry
$$
\begin{align}
\int_{AB,outside}\vec{B}.\vec{dL}&=\int_{-\pi/2}^{\pi/2}\int_{-\infty}^{\infty}\frac{2 (D \sin \theta +1)}{\left(D^2+2 D \sin \theta +z^2+1\right)^{3/2}}dz\, d\theta\\
&=\int_{-\pi/2}^{\pi/2}\frac{4 D \sin \theta +4}{D^2+2 D \sin \theta +1}d\theta\\
&=\begin{cases}
0 & D>1\\
4\pi & 1>D>0
\end{cases}
\end{align}
$$
as expected. This shows the power of Ampere's law.