Why don't main sequence stars continuously expand? I understand main sequence stars become subgiant when hydrogen is depleted in their cores and they start hydrogen shell burning. But I don't understand why this process is divided into two distinct phases, instead of being a continuous expansion as more and more helium builds up in the core. Why is there a sudden point when the star starts "proper" shell burning and begins to expand?
 A: I assume you are talking about the evolution of moderate mass $1.5 < M/M_{\odot} <4$ stars after they leave the main sequence.
These stars have a core that is now made of He, surrounded by a H-burning shell.
The He core starts off with a relatively low mass and gradually accumulates more, due to "ash" from the H-burning shell adding to it.
The core is isothermal because it is not generating energy and is kept hot by the overlying H-burning shell. It can be shown that this equilibrium is sustainable (via a density gradient) until the core reaches the Schonberg-Chandrasekhar limit of around 15% of the total stellar mass. It is this phase that leads to a slow progression of the star to the right in the HR diagram at almost constant luminosity and gradually increasing radius.
As the core mass grows, it reaches and then exceeds the Schonberg-Chandrasekhar limit (in the mass range of stars considered). The core then begins to contract rapidly, releasing gravitational potential energy  that is available to lift the envelope and change the size rapidly on the contraction timescale of the core.
The evolution for lower and higher mass stars is different. Lower mass stars achieve a degenerate core prior to reaching the SC limit. Higher mass stars leave the main sequence with a core higher than the SC limit already.
If you really are talking about main sequence stars then there seems to be a false premise. Main sequence stars do get continuously bigger and more luminous during their main sequence lifetimes, due to the changing chemical composition of their cores. Here for example are the expected trends for a star like the Sun. There is a gradual acceleration once it exhausts hydrogen, but no discontinuity. There is more of a discontinuity for higher mass stars, as I described above, which takes place during the subgiant phase, not at the end of the main sequence.

However, you are correct that there is a relatively sudden transition between core hydrogen burning and shell hydrogen burning (relative to the main sequence lifetime anyway). It is more rapid in higher mass stars; for a star like the Sun, the transition still occurs over a billion years or so. The reason for this is twofold. First, the core is convective - that means that even if the very centre depletes all its hydrogen, a new fuel supply can be mixed in from further out. This means that all parts of the core run out of hydrogen at nearly the same time and once they do so, then convection, which is driven by the energy generation, also stops. Second, the temperature dependence of nuclear reactions is high, and that means that the shell burning reactions are turned on rather suddenly when the shell temperature reaches the ignition point.
Of course there cannot be a gap between core burning ceasing and shell burning starting because overall hydrostatic equilibrium must be maintained, but the transition from one to the other is quite quick because of the two factors above.
A: Please beware, this is rather a qualitative answer without much details or pretense of exactness.
The expansion to a subgiant ist taking place while the helium core has not yet started helium burning and collapses on a thermal timescale. There is the so called mirror principle at work: when the layers below a burning shell are contracting the layers above the shell expand. The contraction of the core and thus the expansion of the whole star stops as soon as helium burning starts.
There are essentially three distinct phases: slow changes (nuclear time scale on the main sequence), a fast phase while the helium core collapses, and again a slow phase (nuclear time scale during helium burning). Note that I’m speaking here of changes in general, not necessarily one specific physical property, like the radius, or the direction of the change e.g expansion or contraction. As an example: when there are two burning shells (helium burning on a CO Core within a helium core with a H-shell on top), contraction of the core will lead to a contraction of the star, while the middle layers expand.
A: Core hydrogen burning is a self-stabilising process. If the density of hydrogen is depleted, the process slows down, generating less heat, so that the core contracts, restoring hydrogen density and the rate of hydrogen burning. This accounts for the near constancy of the luminosity of stars on the main sequence. When hydrogen in the core is exhausted, the helium core contracts, generating heat which then hydrogen burning in a thick shell surrounding the core. The heat generated by shell burning causes the outer layers to expand, and the star becomes a subgiant.
