"Physics breaks down" sounds good, but it is confusing. A better phrasing would be "known physics breaks down." Physics attempts to model reality using mathematics. In this sense, physics has no "laws." In the famous words of Captain Barbossa, "They're more like guidelines." However, we have many of these guidelines which are so astonishingly reliable on the scale that humans live and breathe at that we find the word "guidelines" is too soft. If you are building a skyscraper, you are more likely to make one that stands up if you think of gravity as a "law" rather than a "guideline." Architects who call gravity a "guideline" tend not to be given the opportunity to design multi-million dollar skyscrapers using other people's money!
As such, we should not be surprised when guidelines break down in extraordinary circumstances.
Personally, I would define two versions of this "physics breaks down" concept. One of them is when these guidelines break down in known ways, changing into new regimes. Supersonic flight is one great example. You can learn lots of guidelines about how fluids (like air) flow, but those guidelines break down at the speed of sound in that fluid. We then have supersonic flow guidelines to help us from there. In this case, this breakdown was something observed, typically on the edge of a phase transition. We figured out that supersonic flow followed different guidelines than sonic flow because we flew something faster than the speed of sound, and it behaved differently.
The other, more complete meaning of "physics breaks down" occurs in things like black holes. These are cases where we have never observed the breakdown, but our guidelines lead us in counterintuitive directions. To the best of all of our study, we have never found an experiment or an astronomical observation which suggests that Einstein's theory of general relativity is anything less than a law. Like everything in physics, its just a guideline, but we have not been able to find any situation where it fails.
However, there's a catch. Mathematically, Einstein's theories make strange predictions about extremely massive dense objects. The equations behind it spiral out of control, and you end up with oddities like a "singularity," with 0 volume and finite mass. This means infinite density, and we generally don't think that kind of infinity happens in the universe. So we say "physics breaks down" here because we honestly believe that if we ever get to actually observe this part of a black hole, we will see one of those phase transitions where physics just stops doing the General Relativity thing and starts doing Something Else. We believe this because we are not comfortable with the possibility that such math involving infinities is "right."
Another place "physics breaks down" is on the Planck scale. Planck time and Planck length are very small units of time and space. They're on the order of $10^{-43} \text s$ and $10^{-35} \text m$. They are important for another guideline that has shown to be so effective that we call it a law: Quantum Mechanics. We have yet to see anything on the small scale which suggests that Quantum Mechanics is anything less than a law, just like we haven't seen anything on the large scale to suggest that General Relativity is anything less than a law. However, when we get down to these absurdly small scales, the math of quantum mechanics (in particular, quantum field theory) start to break down, failing to yield predictive answers about what happens at that scale because some of the tools simply stop working. In a very handwavy sense, its like cutting things with a knife, smaller and smaller things, until one day you realize that you're trying to cut something thinner than the edge of your knife. Our metaphorical mathematical knife (renormalization) simply stops cutting at this point.
Of course, we have no reason to believe that the universe "ends" at this scale. It's just the point where all of the guidelines which are so successful that they have earned the monicker "law" fall apart. And in the extreme cases, such as the issues at Planck scale or in black holes, its the mathematics of the theory itself which suggests that the theory falls apart.
And so, we spend countless hours devising new experiments with the goal of plumbing these uncertain regions, with the hope of one day writing new physics which doesn't "break down" at these levels.