I'm currently researching the intermodal distance of transverse and axial modes in optical cavities. But the equation defining the longitudinal separation contains the +- sign, does this imply that both those positions there are longitudinal modes?
$Δν_{transverse}=\frac{c}{(2L)} \left[ \frac{cos^{-1}(\pm \sqrt{g_1g_2})}{π} \right]$
Furthermore, do the higher-order transverse modes of say; axial mode 1, overlap with the positions of higher order axial modes?
I suppose the equation there refers to the special case where $l,m=0$ from the resonance frequencies of two-mirror spherical resonator, which is more generally:
$\nu_{nlm} = \frac{c}{2L} \left[ n + \frac{1+l+m}{\pi} cos^{-1} (\pm \sqrt{g_1g_2}) \right]$
I attach a source in the image link and you can see there that the case where you have a stable cavity is only when $0 < g_1 g_2 < 1$. Therefore the sign only takes care of giving a positive value in the end ($cos^{-1}$ has values between [-1,1]. Also remember that the solution should be real so that also considers the sign convention above.
Regarding the second part of your question of whether the modes overlap: In general if the modes inside of the cavity satisfy the stability condition, then the excited modes inside of the cavity are eigensolutions of the wave equation inside of the cavity, and modes which are orthogonal to each other (such as axial and longitudinal modes) would not overlap as they are resonant in different axes, if that is what you meant. If it is rather whether several modes can exist inside of the cavity then I would generally say yes, they would "overlap" in the sense that they are valid eigensolutions and can therefore exist inside of the resonator.
