There are many questions on this site about the quantum eraser, but I think mine is not quite answered by any of the other answers on the topic.
Here's the setup:
My understanding of this experiment is the following. If a stream of photons is passed through a double-slit one at a time, but then passed through the BBO crystal, the photon is marked with which-way information due to the nonlinear interaction (basically the act of destroying a photon and making two new ones represents a measurement which collapses the wavefunction). After this the entangled pairs are split; one member of the pair is sent to a detection screen and the other member of the pair is sent into this setup with mirrors and beamsplitters and finally onto some click detectors $D_1, D_2, D_3, D_4$.
Based on this description, it seems obvious that if we look at the photons at the detector screen $D_0$ whose entangled pair photons hit $D_3$ or $D_4$, we'll see no interference pattern: in these cases we have which-way information and there should be no reason for interference.
What confuses me is what $D_0$ registers when you look at the photons whose pair photon hits $D_1$ or $D_2$. I understand that, due to the setup, the two paths have been recombined in the lower portion of the experiment, so that a click in $D_1$ or $D_2$ does not tell you which slit the photon went through anymore. But the upper photons are still marked with this information, so why should we recover an interference pattern? In other words, based on my understanding, the BBO crystal should prevent us from ever seeing an interference pattern.