I'm going to take a whack at answering this. While these questions may seem naive, they are still the basis of research that cosmologists are doing today. But I'll try to keep my answer to a sub-grad student level.
Let's start with space and the vacuum.
First of all, space is not infinite in size. If it were, everything would have cooled to 0K a long time ago and the probability of finding another galaxy cluster (let alone multiple millions of them) within our visible horizon would shrink to very near zero unless there were also an infinite amount of matter (that would be bad).
Now that we've established that, space and the vacuum are not the same thing. A pure vacuum is a volume that contains nothing. While this includes an absence of EM radiation, usually we use the term to describe an absence of matter. There are no known natural or artificial pure vacuua. Every volume (within reason) has some matter in it. For instance, there is 100 times more air in a random volume from between the stars of our galaxy than there is in the same volume from between two galaxies. But this is just trivia. Space, on the other hand, is (as referred to in physics) the fabric of the 3 geometric dimensions. If you picture the x-, y-, and z-axes and you extend them over all the universe, that is space. Space is not, however, the basic building block that allows matter to exist nor is a point in space. A point in space has zero size. It is just a coordinate. You can no more ask what the point right next to it is than you can ask what number is right next to pi (if anyone says it's pi+1, I will slap them).
Moving on to your second paragraph: Space is not at absolute zero. The accepted temperature of space is around $2.7K$ ($-270.45^\circ C$). As for laying in another dimension, there are a number of cosmological theories surrounding this. Without going into far too much detail, I will just say that there are no higher geometric dimensions and to say more would be well above the scientific limit for this post.
Now the fun part. When we say the universe is expanding, that is usually in reference to itself. The universe is expanding within itself. There are again some cosmological theories that provide alternate media for it to expand into, but at this level of understanding, what I said will hold. Picture, if you will, an ant on the surface of a balloon and suppose that ant has no concept of anything but the surface of that balloon. Then imagine someone starts inflating the balloon. From the ant's perspective, the balloon has no boundaries; things are just getting further and further away from it and the amount of space to walk around in is growing. The ant, thus, knows that the balloon is expanding. The analogy doesn't completely work; we are believed to be the interior of the balloon, there isn't a 4th geometric dimension, and the universe isn't as curved as much as a sphere, but for explaining expansion, it works.
There is a rate of expansion. All points in space grow apart from each other. However, gravity works against that; it holds objects together. The result being that it looks very much like all heavenly bodies are moving away from us. However, because we have objects on all sides of us, we can use the milky way as the reference and see that things are moving away in all directions. We can also tell that objects farther from us are moving away faster. This is because there is more space in between and so, more expansion.
I may have misinterpreted some of your questions or comments. I hope this helps somewhat but let me know if I left anything out.