This is only true if the source is perfectly collimated, and that is never the case. If the source is not perfectly collimated, then it will produce a fan of rays and each of these rays will appear at a different place of the detector.
One such ray, for example, will interfere like this:
Image source.
A ray sent slightly downwards as shown in the image will arrive, after the interference, at a detector placed to the left of E. Moreover, unless the placement of the mirrors is exactly right, these two rays will accumulate slightly different phases along the beams, so they are unlikely to both have the same relative phase between the beams and therefore the same type and amount of interference. The interference conditions will therefore change across the detector screen, causing the interference pattern.
The interferometer can therefore be adjusted so that there is a clearly-defined interference pattern on the screen, or such that the interference is perfect throughout (which requires much more careful alignment). In particular, it is often advantageous to keep the alignment such that a number of fringes show on the screen, which can make it easier to count these fringes as they slide past the screen.