To understand this experiment, it is helpful to stop thinking in terms of the wave-particle duality and which-way. If you consider a photon to be a wave packet, or a packet of a superposition of waves, which describes reality much better, the experiment gets quite straightforward. See my video: https://m.youtube.com/watch?v=e-uopGrghFQ. The bottom line is that it is not the presence of the which-way in itself that determines the appearance of the interference pattern, but the presence of a phase shiftingphase-shifting component (the beam splitter) or direct detection. The interference pattern itself is a post-selection effect: It is caused by the combination of the phase shift of one of the waves, resulting in a hit probability on the detectors that depends on the phase difference, thus the difference in path length, thus the angle at which the photon left the bboBBO-crystal, thus the location where it will end up on the detector screen.