"Infinite" for me does not make much sense in physics. It is a nice mathematical tools, you need it to make calculations; but for real things I prefer "very large". I have seen many "very large" things; I have never seen anything infinite.
Anyway, more seriously, we use all the time infinite dimensional spaces, they are useful. From the example of "particle in a box" by >JSorngard:
You want to keep your particle in one eigenstate, against external disturbance. It can excape going to other eigenstates. Those, in theory, are infinite, so to calculate the probability for your particle to fall off from your favourite eigenstate, you sum the transition probability to each of all those (infinite) states. And it works!!
In practice there are not really infinite eigenstates; the particle is confined by some actual physical trap that is finite in size and can hold only finite energy. But incredibly often you can disregard this finiteness, as the infinite sum is almost identical to the (very large) sum of the actual eigenstate.
Another argument in favour of usefulness/reality of infinites is about notation: we use infinite things to define & manipulate a normal object.
Example is the Taylor expansion of $e^x$:
It is an infinite sum, its useful, don't give rise to anything nonsensical.