Differentiability of the EM fields, like for many other quantities introduced in Physics, is not a property of the world but it is part of the mathematical model we find useful to describe the world.
As such, it is a property whose validity may be judged on an experimental basis. Until the model is in agreement with experiments, the property is valid. As soon we find a significant departure from the experiments we should be ready to change our model.
Actually, for EM fields we know that continuity properties may hold only on a coarse grained scale where the spatial average property of any real measurement allows to deal with smoothly varying quantities. However at the level of a microscopic QFT description a classical EM field emerges only when we can neglect the effect on the local values of the fluctuations of the underlying quantum fields.
Notice, that even much before QM or QFT entered into the toolbox of physicists, the smoothness of macroscopic fields was considered as a consequence of averaging over spatial regions large with respect to typical atomic length-scales but small with respect to the typical length of variation of the fields.
This is exactly the analog of a classical trajectory. We know that it does not exist at the QM level, still it is a very good approximation, with all possible smoothness, if we have to deal with planetary orbits.