What interesting physics problems can't be solved because mathematics is not developed enough? I'm curious as to what sorts of physical problems to which we don't have an answer, because we haven't developed the right mathematics yet (or advanced-enough mathematics). 
Related to this question on the Math StackExchange: In what ways has physics spurred the invention of new mathematical tools?
 A: Lots of real world physics problems contain a huge number of variables and therefore have very complicated equations describing their associated dynamics. An example of this is non-linear partial differential equations, which are notoriously difficult to solve. As an example, take the Navier-Stokes equations. These are real physics equations which describe the motions of fluids and their physical characteristics.
Nobody has yet solved the problem concerning global existence and uniqueness of these equations, and in fact it is one of the millenium prize problems owing to its difficulty. Relating to your question, we effectively have an equation describing a physics phenomena for which the mathematical tools are not yet sufficient to prove is well defined and as such it is difficult to talk about how precise the extracted physics is regarding the solution of this equation.
A: Arthur Suvorov gives a nice comment, I am just going to give a list of a few specific physical problems I can think of from the top of my head.


*

*Yang Mills existence and mass gap (Millenium Prize) and generally the problematic of rigorous definitions and constructions of quantum field theories

*Navier Stokes equations and smoothness (also Millenium) - it's not only Navier Stokes, but the whole topic of turbulence that bugs even MHD people etc.

*Exact solutions to Einstein equations - we have exact solutions for supersimple situations, but as soon as we would like to generate physically plausible solutions for situations only slightly more complicated, as an axially symmetric stationary space-time (see Ernst equation), the math just does not provide the tools. This applies generally to soliton generation techniques in various fields.

*The question of ergodicity of general systems - the "arrow of time" is considered in the most general case a consequence of the ergodic hypothesis. But it has been proven only for a very restricted class of systems. The whole "arrow of time" discussion might get a kick out of a more general truths about ergodicity or even a really rigorous treatment of the fact of complexity rising against the pull of entropy growth. 

*Finding non-perturbative solutions to QFT problems (see a popular account of a recent neat development)

*Equation of state of the centre of a neutron star - this is connected with the previous question, because the conditions in the center of a neutron star are such that a non-perturbative treatment of e.g. a quark-gluon plasma would be needed. The ignorance of such equations of state is a major obstruction in understanding phenomena such as black hole formation. 

*Good definedness of the Feynmann path integral formulation - you can see a review here. I am not sure what physics can be lurking there, but I believe that a rigorous definition might provide a hallway to new knowledge.

*Much more - basically all of the speculative beyond-standard theories are a mix of taking physically plausible steps with solving mathematical issues which spring up at every of the steps. There will surely be a long list which I don't feel qualified to even start.

