I think it's because we sometimes talk about quantities that are not the result of measurements, but rather conceptual entities. For example, when you pump a laser, at some point you go through a situation where the upper level population equals the lower level population, prior to inversion. At that moment, you could say the temperature is effectively infinite. You will never stick a thermometer in there that reads infinity, but the concept of infinite temperature is encountered. Or, one might ask if the size of the universe is infinite, even though no size measurement will ever come out infinity.
Similarly with near-zero results. You can talk about "absolute zero" as a concept for temperature, though Nernst's law tells you that you'll never actually encounter it. So if you say "the temperature must be finite", you might mean you can't get a zero result, though I agree that's an imprecise use of the mathematical meaning of "finite." "Nonzero" would suffice, and be more accurate, it's some sort of conventional misnomer, like "blackbody radiation" or "degeneracy pressure."