I do not know any fundamental minimum of Peltier elements operation temperature.
However, there are serious technical issues: as long as Peltier effect relies on an interaction between electrons and phonons, there should be enough phonons to interact with. Decreasing temperature dramatically (though quantitatively, not qualitatively) changes effectiveness of the elements. As long as heat pumping should overcome heat produced by electron current which produces this pumping, going to low temperatures is a challenging task.
To sound more scientific, I've found this relatively new paper where Peltier effect is discussed for some rather standard system and where curves at room and liquid nitrogen temperature (which is low by human standards but pretty warm from cryogenic point of view) and its effectiveness may be found (see Fig. 3). As can be seen from the curves, Peltier element is able to give $\Delta T$ around few degrees at nitrogen temperature. So, it stil works at 77K but definitely far less effective than at room temperature.
With numbers given on graph I can hardly imagine the cascade which will do these 77K out of room temperature.
In this paper authors claim that effect may be observed at 6K, but the numbers they give show that this effect may be hardly used in practice.
To conclude, it seems there is no definite theoretical limit, but practical limit is around -100C