do units have to make sense in Physics?
Absolutely. Unless one uses natural unit system, where some value is the ratio of a quantity with some known constant. I.e. imagine that you are expressing speed in terms of $c$ - spaceship is moving at $1/2~c$, or - sphere has $200~e$ charge. You can supply absolute value for $c,e$ and you will know exact measurement value in SI, CGS system or whatever. Or you can simply assign $c=1,e=1$ and just express every measurement as the ratio to some base unit.
if you can cancel out all units, then you can give it any units you want in the end
Really ? So you say, that if I measured that concrete lightning-bolt has struck with a charge of $1000\times$ (no units) of an average lightning-bolt power in terms of charge,- then you can say that it has struck with $1000\times$ of average milk fat content ? What's the point of such nonsense ?