The total gravitational force on the whole rope will be $mg$, as you said.
You can get this by integrating the net force acting on each infinitesimal mass element $dm$ over the length of the rope. Let downwards be positive, and the origin be at the top of the rope. Assuming uniform linear mass density, $dm = \lambda\, dx$. An element at position $x$ will experience $g\,dm=g\lambda\,dx$ (its own weight) and on top of that a tension $T(x+dx)$ downwards (from the element below it) and a tension $-T(x)$ upwards (from the element above it).
For the total gravitational force on the rope, just integrate $\int_0^Lg\,dm$, and you get the expected answer $mg$.
To find tension, we need to do more work. Since the rope is at equilibrium, $g\lambda\,dx + T(x+dx)-T(x) = 0$. Rearranging:
$$g\lambda = \frac{T(x+dx)-T(x)}{dx} = \frac{dT}{dx}$$
which you can then integrate and solve for the tension (after applying some logical boundary condition).
So conceptually, the total gravitational force on the rope is nonzero. However, it is at equilibrium because it is suspended (and presumably fixed, so there is a second external force acting on it.) Each small mass element is at equilibrium because the tension in the rope balances the mass element's weight.
