As Pangloss points out, this is surprisingly non-trivial!
The sort of manipulations you are talking about are part of a quantity calculus. This is not to be confused with Newton's and Leibniz's Calculus. A calculus is just a method for computation. This is a method of manipulating quantities which yield "the same" quantity with different units.
Your method of "canceling" the units is the normal way of phrasing one of the rules that can be applied in quantity calculus. And, we show very informally that it works, because nobody has found a way it doesn't work. It's anthropocentric, but its the truth. Like so many systems, we can do the things we can do in that system because we evolved countless systems that didn't work, and this one did.
A major aspect of answering your "why" question is the axiomization of unit conversions. This is the process of "turning the crank" on these unit manipuations to produce all true unit conversions. In theory, if you could do this, you could start to make statements about how they work and prove something more profound.
As it turns out, we cant do that. We do not currently believe that units "in general" can be axiomized in this way. However, we are confident that the basic scientific units (meters, seconds, etc.) can be treated this way. Perhaps they are a special case, perhaps they are a general rule.
Metrologia keeps their papers behind a paywall, but a particularly interesting paper might be On quantity calculus and units of measurement. This discusses an interesting pattern where our basic scientific units can be treated as quantities in their own right (i.e. "ft" corresponds to "1 ft"), and thus admit algebraic operations like division. However, it points out that, in general, this is not a valid construction.