Interestingly, I think an argument can be made that the very definition of a "beautiful theory" has changed significantly over the past century.
When I read the old physicists, starting in particular with Maxwell, a beautiful theory is simply one that predicts things about the universe using some very sparse set of assumptions -- a set so sparse that there is no way those predictions could have been "programmed" into the original set of equations.
I think the first physicist who nailed truly nailed this prediction-from-simplicity concept of beauty in the form of mathematical expressions was James Clerk Maxwell. He started out trying to model everything that was known at that time about electromagnetics. He did so by using a variety of models that by stated intent were highly physical (e.g. flow or "flux" lines) and easily visualized since Maxwell believed that if he could not capture everything within such a model, he likely was misunderstanding something. Thus his use of physical models was in no way intended as some kind of crutch, but rather as a way of forcing himself to validate his understanding in a way that did not rely on glossing over details. (For anyone interested, this mostly forgotten phase of Maxwell's short but amazing intellectual career is beautifully described in Basil Mahon's The Man Who Changed Everything.)
What happened next is remarkable and worth noticing, because I think it captures the concept of a mathematically beautiful theory so well. Maxwell realized that all of his mechanical models could be replaced -- summarized and fully captured -- in the form of a small set of relatively simple differential equations. Interestingly, his equations were not all that well received at the time, since many people had followed and liked his more mechanistic explanations very much. But Maxwell's point was profound: All of the seeming random complexity of a very diverse set of electrical, charge, and magnetic phenomena could be seen through his equations [1] to be different views into a single and unified whole. When people talk nowadays about "unified field theories," it's important to recall that Maxwell unequivocally was the first to create a theory that unified two seemingly quite disparate forces, electrical and magnetic. (Newton in contrast recognized the unity of phenomena that were all related to a single force, gravity -- a great accomplishment, but different in type from what Maxwell did.)
The equations that Maxwell produced were sufficiently power to lead him to recognize that light is a form of electromagnetic radiation -- an incredible insight at that time, and completely unexpected. It is also an example of their beauty, that is, of their ability to predict real phenomena based on a very sparse set of symbols and concepts.
Once Maxwell got the ball of mathematical beauty rolling, things really started popping, albeit a few decades later. (Maxwell was unbelievably ahead of his time.) First, there was that chap named Albert Einstein who figured out an extraordinarily small equation (how he found it is a story in itself) that equated lumbering mass with dynamic energy, then another theory by the same fellow linking conceptually curved geometries to the force of gravity. I think Paul Dirac gets the next nod for coming up with a compact equation (aptly called the Dirac equation) that seemed to indicate that everyone had overlooked half of the forms of matter possible. Ouch! Even Dirac didn't really believe his own result. But in two years, he was proved correct. Now that's mathematical beauty: A compact equation that doubled the number of particles now known to exist in the universe!
I have to add one other more purely mathematical example of the predictive power of simplicity: $\mathbf{C}$, the complex plane. While nowadays $\mathbf{C}$ is presented primarily as a boring given of basic mathematics, it actually took centuries of insight and discussion before the computational and conceptual power of this seemingly simple idea was fully recognized. Quantum mechanics is built almost entirely around $\mathbf{C}$, for example. Why? Because the phase relationships (angles) that are so trivially and elegantly expressed using complex numbers are "just what the doctor ordered" for the wavelike phase relationships that are fundamental to quantum mechanics.
Along those same lines, Maxwell's equations initially were based on $\mathbf{H}$, the quaternions. This set of ideas, which are a generalization of complex numbers, were discovered literally because William Rowan Hamilton -- another man who like Maxwell was decades ahead of his time -- at one point focused his attention on finding a way to generalizing complex numbers for use in three-dimensional spaces. And even though his quaternions fell by the wayside, it is from $\mathbf{H}$ that modern vector mathematics, including dot and cross products, arose as a sort of higher-dimensional generalization of selected components of quaternions.
Math itself thus can represent a type of beautiful theory in its own right, in the sense that throughout the history of mathematics, relatively simple ideas there would turn out to have massive predictive power when applied to the physics of the real world.
So, that's the "old school" definition of mathematical beauty, and it's pretty consistent in what it's all about predictive simplicity.
Now, jump ahead nearly a century to the last 30 or 40 years. The single most common topic of conversation in which the concept of "beauty" is promoted as fundamental to its activities is string theory. If you look around you can find all sorts of praise of how string is "very beautiful" and even that it "must be right because it is so beautiful." What is meant by this use of the phrase?
No matter whether you like or dislike string theory, this use of "beauty" for a mathematical theory cannot be the same as the one that applies to Maxwell, Einstein (twice), Dirac, and other greats from the late 1800s and early 1900s.
Why?
Because string theory does not predict anything. After almost forty years of trying, not a single prediction of a single physical effect has ever come out of it, although I think there have been some attempts in recent years to try to come up with some potentially testable predictions.
Also, string theory is emphatically not "simple," since if anything it goes in exactly the opposite direction: Much of it uses extraordinarily complex mathematics intended to describe the extraordinarily diverse set of vibration phenomena possible if you postulate a 9 or 10-dimensional space.
So what is meant by "beauty" in this case, if not (most definitely not!) predictive simplicity?
To be honest, I've always wondered that myself. The best explanation I can come up with is that there is a hope for simplicity to pop out somewhere from the enormous spaces and enormous possibilities possible from higher-dimensional spaces. So perhaps the idea is that someday, somewhere, a truly simple vibration model will be found that neatly explains everything.
But for me, no. I cannot call such large masses of cryptic equations "beautiful" without feeling like I've betrayed the spirit of simplicity of the early fathers of physics. They truly believed that simplicity was there to be found -- and most importantly of all, they then went out and found it.
Now that's beauty.
[1] Maxwell's original equations emphatically were not the four equations that now bear his name. Maxwell originally developed about 20 equations with 20 variables. Even more oddly, they were written using quaternions. These are four-dimensional entities that can be thought of as generalized complex numbers for which the $\mathbf{i}$ value defines a unit vector in a three-dimensional space, rather than a single direction in a two-dimensional space. It was not until over 20 years later that Oliver Heaviside reformulated Maxwell's equations into four highly compact -- and thus considerably more beautiful -- vector-based equations. The one and only reason these massively updated and reduced equations are not called the "Heaviside Equations" is because Heaviside himself was adamant that all credit for the thinking contained within them should remain with Maxwell.