Should every physical problem formulated as a differential equation have a mathematical solution? I encountered the following statement in Boyce's Elementary Differential Equations and Boundary Value Problems : 

Not all differential equations have solutions; nor is the question of existence purely mathematical. If a meaningful physical problem is correctly formulated mathematically as a differential equation, then the mathematical problem should have a solution. 

Is this true? 
 A: Maybe there is more context that qualifies this statement, but taken as is, it's completely false. In general, when we talk about existence of solutions to a differential equation, we're talking about existence given a certain set of boundary conditions. It's perfectly possible, in practical real-world problems, that we can have constraints on the boundary conditions, and if our boundary conditions don't satisfy those constraints, there is no solution.
For example, I could write down a differential equation representing the free motion of a body in a viscous medium. I could then specify the following boundary conditoins: at $t=0$ its velocity is zero, and at $t=t_f>0$ its velocity is nonzero. There is no such solution.
In physical problems where we specify the initial conditions, we want not just existence but uniqueness of solutions.
Less trivially, we can have examples of inconsistency or indeterminism (solution exists, but is not unique) in physical problems that come up in interesting, "meaninful" contexts. Examples include Norton's dome, naked singularities, and the Novikov consistency principle.
A: In the best of all possible worlds we know the laws of physics perfectly and can write them as differential equations (or something similar). But we do not live in that world. Instead we create models of the physical world that may not correspond to the actual laws (due to ignorance or just approximation). Good models give informative predictions: there is a mapping between what happens in reality and the model that is close to a bijection, so we can use the model to predict physical responses. How close it has to be depends on the application.
Now, physics as far as we know never fails to produce a "solution" of what the future state of the world will be. But models clearly can fail at this, even models that are accurate in large domains. Depending on the application this might disqualify them - only use models that use differential equations that have solutions! - or be OK if model failures occur far away from the problems of interest. When the Schwarzschild metric in GR predicts geodesics that end up in a singularity it may not be a problem if one is working on orbits or stuff going on far away from the central mass.
The issue isn't whether physics "runs" on differential equations that always have solutions, but all about what properties a useful model should have in the case at hand. In short, it is not about differential equations but modelling problems well.
