There are many different versions of the multiple-universe idea in physics. Scientific American has a nice (skeptical) article on the topic this month, "Does the multiverse really exist?" by the well known relativist G.F.R. Ellis. Some of the possibilities discussed include the many-worlds interpretation of quantum mechanics, chaotic inflation, the string-theory landscape, and cyclic models such as ekpyrotic models and Penrose's conformal cyclic cosmology. You can find information about specific ideas from this list on Wikipedia, etc. There is not likely to be a generic answer to your question that will encompass all of these cases.
One thing you have to watch out for is that it is not necessarily easy to come up with a suitable definition of "universe." For example, you could say that it means the set of all events that a given observer O, who is assumed to be immortal, could in principle observe, subject to general relativity's restriction to the propagation of information at no more than the speed of light. The accelerating expansion of the universe implies that there is a cosmological horizon beyond which O can never see, so by this definition of "universe," O's universe has a boundary. With this definition, if you have two observers, O and P, there can be events that are in O's universe and also in P's universe, but other events that are in one but not the other. You could say, "Oh no, I don't like that definition, it's goofy. I want the whole universe to be included, not just the part that a particular observer can ever observe." But then you have the same philosophical problem as in other incarnations of the multiple-universe idea, which is that you have to talk about things that can never be observed, even in principle.
I will give one example where it seems to me to be valuable to discuss multiple universes.
History offers several examples of cosmological models that were wrong, and that with hindsight we can say should have been considered implausible at the time, because they required fine tuning. Newton envisioned an infinite, homogeneous cosmos so that despite the force of gravitational attraction, the force on any particle would cancel out by symmetry, thereby allowing the universe to exist forever. Einstein had a similar model in which the cosmoogical constant was exactly sufficient to balance gravitational attraction and keep the universe static. Even before Hubble discovered cosmological expansion, there was a problem with both of these models, because they were unstable, and required fine-tuning. In the Newton model, any deviation from homogeneity, no matter how small, causes a snowball effect. Similarly, Einstein's cosmological constant had to be perfectly tuned, or else there would be a vicious cycle of accelerating expansion or collapse. The lesson from history seems to be that we should not believe in cosmological models that require fine-tuning.
But the current consensus model of cosmology has multiple fine-tuning problems. There is a flatness problem, and also an entropy problem. Inflation is supposed to fix the former, but inflation has a lot of problems, one of which is that inflation is in some sense improbable. To define the notion of its probability, you pretty much have to start talking about the set of all possible universes, and asking how many of them have inflation that could lead to a universe such as ours. This is similar to the Einstein fine-tuning problem, where you basically want to object because the perfectly tuned value of the cosmological constant seems low in probability. You can't talk about probability unless you have a sample space, and here the sample space is some kind of multiverse. If you refuse to talk about the multiverse, then you've closed off an avenue of argument that has been historically fruitful.