One fundamental problem in mathematical physics that has to my knowledge not been resolved, but I might be mistaken since I haven't been working in this particular field for already 5 years and 5 years is a long time, is the following:
In macroscopic bodies, there are several laws existing describing transport phenomena like heat conduction. They are described by precise mathematical laws which are well established experimentally. A well-known one is Fourier's law:
$$c\frac{\partial}{\partial t}T(\vec{r},t)=\nabla \cdot \left(\kappa \nabla T(\vec{r},t)\right) \; ,$$
where $T(\vec{r},t)$ is the local temperature, $c=c(T)$ the specific heat and $\kappa=\kappa(T)$ the heat conductivity. There is however no rigorous mathematical derivation of this equation for any classical or quantum model with a Hamiltonian microscopic evolution. This problem is related to the question wether deterministic microscopic systems can fully explain the behaviour of macroscopic matter. (Remember this question, well basically, we have no rigorous proof that it's all just probability applied on huge amounts of microscopic particles. At least not for out of equilibrium processes, which transport phenomena are.)
Now, I don't expect anyone but mathematical physicists invested in this particular field to loose any sleep over it. We have a good heuristic understanding of how things work and there's nobody seriously doubting that the explanation of macroscopic phenomena lies in understanding the underlying microscopic phenomena (except maybe some crackpots and fringe physicists). Still, a mathematical derivation could bring us a deeper understanding of why it works.
The Green-Kubo relation gives $\beta V \int_0^\infty d\tau \left\langle J(0)J(\tau) \right\rangle$ for the transport coefficients. The time autocorrelation function can be computed using the closed time path formalism.