Consider the particle in a box problem in QM. The crux of the reason why QM is able to explain the physical phenomenon is not just the theory but also able to impose boundary conditions which eventually result in quantization. Now in the particle in a 1-d box problem, the wave function is assumed to be zero at the boundaries. It has been said that it is imposed, so that the wave function is continuous. Okay, but what about differentiability? In order for the wave function to satisfy Schrodinger equation,we also need differentiability right? Okay if we assume only left (from one side) derivative to exit, we could have as well assumed only left continuity (from one side). For continuity, we assume it should be from both sides, but for differentiability we need only one side? They also say the slope also must be continuous. I don't see any rationale behind these quantizations!
Differentiability of the wave function is only required for finite changes in the potential. If the your potential is infinite (as it is outside the inifinitely deep potential well which you describe) the Hamiltonial is ill-defined anyways.
An other case where you can have an infinite potential is if you have a $\delta$-distribution as a potential, there again you will find that the wave-function must be continuous but not differentiable (the difference between left-side and right-side derivative is given by the strength of the $\delta$-potential in this case).