Or was all this done solely by mathematical deduction based on models we built around what we know about nuclear fusion and fission?
This is pretty much it. Generally, stars don't evolve fast enough for us to see it happening*, so we're left with a problem more like archaeology: we see stars in various different states, and it's up to us to make a model of how they all fit together. Our "experiments" are numerical.
That said, and despite all the open problems in stellar physics, the model we have is quite simple and works really well in making sense of the various classes of star. The basic assumptions are basically that stars are self-gravitating spherically-symmetric balls of opaque plasma in hydrostatic and local thermal equilibrium. This already tells you that the centres are hot and dense enough to initiate nuclear reactions. Adding them into the equations gives the main source of long-term pressure balance. You also find that in some places, the plasma is unstable to convection, so you have to tack in a model for that too. From there, it's just a case of letting the models evolve (which is mainly because the reactions change the chemical composition), and you find you can reproduce most major stellar types, and therefore conclude, say, that the Sun will become a red giant. The quantitative predictions might be a bit off, but qualitatively everything fits together.
There are, however, some stars out there that confuse this picture. The most serious complication, in my opinion, is that stars are often close enough to other stars that they interact, and this can trick you into misunderstanding what's going on. For example, you can read up about the Algol paradox or the blue stragglers, both of which would defy conclusions from the single-star picture.
Stellar winds are another thing that complicate the picture. What are we to otherwise make of Wolf-Rayet stars and subdwarf B and subdwarf O stars? They don't come from the simple picture unless we add substantial winds to the model, and our formulae are mostly still empirical.
Plenty of other details remain uncertain, but the simple single-star picture, based on reasonable physical assumptions, explains most of what we see, from which we conclude that it must be at least mostly correct.
*An interesting exception is supernovae. Not only do we see the supernova evolve, but we're now able to look at images from before the supernova to determine which star was the progenitor.