In figure (a), the voltage is continuous but the time derivative is not; the capacitor current would discontinuously change sign from positive to negative.
In figure (b) however, the voltage is discontinuous. It is typically said that the voltage across an ideal capacitor is continuous since, for the current to exist, the time derivative of the voltage must exist.
However, in the context of distributions, then for example, the voltage across the ideal capacitor can be the unit step $u(t)$ which implies an impulse of current
$$i_C(t) = C\frac{d}{dt}u(t) = C\delta(t)$$
Mathematically, this is sound. Physically, this is absurd since the assumptions upon which this result is based are invalid.
The ideal circuit theory approximation holds only when we can ignore electromagnetic effects which is to say, we assume appropriately slow changing currents and voltages such that, e.g., self inductance can be ignored.
An 'infinite' rate of change would 'infinitely' violate that assumption, i.e., we would have to account for electromagnetic radiation which involves adding 'parasitic' circuit elements into the equation.