How do we know the size of the universe? Ok, from astronomical observations we can tell that the observable matter is separating - so rewind the clock about 13.7 billion years and it was all at a single point.
However, how do we distinguish between the following two options:  


*

*Universe is expanding  

*Matter distribution is increasing into infinite void


Clarification 1:
(My notion of) The traditional notion is that all time/space/matter was created at the instant of the big bang.
I.e. BB was inital conditions of: $t=0$, $V=0$, $E=very big$
People say "the universe is expanding", rather than "observable matter is separating".
Why is this?
How do we know that the big bang event wasn't started by all matter condensed at a single point within a larger (otherwise empty) universe?
How do we know that BB wasn't: $t=0$, $V(universe)>0$ but $V(matter)=0$, $E=very big$?
 A: I will assume that E in the question is energy density.  First we have to distinguish between two sizes:


*

*size of the observable universe = current proper distance to particle horizon = current proper distance to infinite redshift, and

*size of the whole universe.
Assuming trivial global topology, the size of the entire universe is infinite if space is flat or open (hyperbolic), and finite if space is closed (spherical).  To note, if the magnitude of the current density parameter for curvature $\Omega_k(to)$ is smaller than a certain threshold, it is impossible to know whether the universe is flat and infinite or closed and finite. (Vardanyan et al (2009) How flat can you get?)  Clearly the total size of an infinite universe was infinite at t=0, while the total size of a finite universe was zero.
Now, the alternative view of matter travelling into pre-existing empty space (Minkowski spacetime) is precisely the model proposed by Edward Milne in 1932.  For that precise model to be valid we have to assume:


*

*either that GR is not applicable to cosmology (Milne's original position),

*or that the universe is gravitationally empty at large scales (an unlikely but still open possibility).  
Now, if you stay within GR, then matter distribution determines spacetime, so that a non-neglibible matter density implies a spacetime different from Minkowski (so that if the matter moves, the spacetime changes over time). Having cleared that, as I said in this answer, any spacetime described by the FLRW metric, where objects are static in comoving coordinates and redshift is due to expansion of space, can also be described, via an appropriate change of coordinates, by a spherically-symmetric (SS) metric, where objects move along radial timelike geodesics and redshift is due to positional (gtt) and Doppler factors. Since both metrics describe the same physical system, they are observationally equivalent in all respects.
Two notes on the equivalent SS metric:


*

*usually (but not in the Milne case) has a cosmological horizon (gtt & grr -> $\infty$), located at the Hubble distance in flat FLRW models,

*is static (gtt & grr do not depend on time), only in the empty (Milne) and lambda-vacuum (de Sitter) cases,

*even in the next simplest case, the flat matter-only FLRW model known as Einstein-de Sitter model, gtt & grr cannot be expressed algebraically in terms of non-comoving time (t' below), as I mentioned in this answer. 
I expanded this below to practice a bit of LaTex:
Any homogeneous and isotropic spacetime described by the FLRW metric in terms of a set of coordinates $(t, r, \theta, \phi)$:
$$\begin{equation}
ds^2 = - c^2 dt^2 + \frac {a(t)^2 dr^2} {(1 - k r^2)} + a(t)^2 r^2 d\Omega^2
\end{equation}$$ 
where:
t = comoving time = proper time for all observers at constant $(r, \theta, \phi)$
r = comoving radial coordinate = radial coordinate enclosing constant proper mass
$d\Omega^2 = d\theta^2 + sin^2(\theta) d\phi^2$
can be described by a spherically symmetric (SS) metric in terms of a different set of coordinates $(t', r', \theta, \phi)$, where the ' does not mean a derivative:
$$\begin{equation}
ds^2 = - c^2 gtt(t', r') dt'^2 + grr(t', r') dr'^2 + r'^2 d\Omega^2
\end{equation}$$ 
where:
$\Omega$, $\theta$ and $\phi$ are the same as in the FLRW metric, and obviously
$r' = a(t) r$
Expressing $t' = f(t, r)$ and its partial derivatives as $ft$ and $fr$:
$$\begin{equation}
grr = \frac {1} {[1 - k r^2 - \left( \frac {r} {c} \frac {da(t)} {dt} \right)^2]}
\end{equation}$$ 
$$\begin{equation}
gtt = \frac {grr} {ft^2} (1 - k r^2)
\end{equation}$$ 
The problem lies in expressing gtt & grr in terms of $(t', r')$, which can be done only in the Milne and de Sitter cases.
A: 
When you overlap a point from figure A with corresponding point from figure B, you will see that every other point has moved away. 
You go to any point in the universe you will see that things are moving away from you. It is like you put ink dots on a balloon and when you blow the balloon distance between the ink dots will increase. 
If matter distribution were expanding into infinite void [point (b)] there is a question what is causing that expansion? As gravity pulls things there must be some force that is stronger than gravity that is pushing things away. But we don't know any such force. So it must be universe expanding.  
Watch "A universe from nothing" by Lawrence Krauss. 
NOTE: I would have done better job at explaining but I don't know much GR. :(
A: What I think is being asked here is about the alternatives theoretically (and experimentally) consistent with the cosmological solutions in GR. Also some history of terminology.
First it would be useful to distinguish between the Observable Universe and the Universe. There are so many theories around that this distinction will help clarify communication (and understanding). Of course the Observable Universe is what is observed and deductions about what is true outside of that (from some theory) are not going to be astronomically measurable.
Now to parts of the question:

People say that "the universe is expanding" rather than "observable matter is separating".

If the Hubble galactic expansion had been discovered in the decades before Einstein's General Relativity (GR), then perhaps the second phrase would have been more common initially. However GR also contains Cosmological solutions intended therefore to describe the Universe (and not just the Observable one). At around the time of Hubble's discoveries the Lemaitre-Friedmann solution was gaining ground in which GR is seen to provide a specific expansion of the Universe from a t=0 singularity. So the early and essentially continual match between theory and astronomical observation has resulted in the first phrase being used as well.

How do we know that BB wasn't: t=0, V(universe)>0 but V(matter)=0, E=very big?

Strictly what describes the t=0 solution of GR is outside of GR. However leaving that point aside there are some comments.


*

*V(universe) =0 or >0


This really asks what the singularity can be modelled as in GR. In the days when the closed universe model was prominently discussed it was natural to talk about universe radius and origin at a point for the singularity. However, and especially when the flat or open models seem more observationally accurate, those earlier assumptions do need to be re-considered in accounts. For a flat model say, then the singularity could be an E^3 - much different from the "point" often discussed (although other geometries may be possible too).
However the density of the observable universe's matter at this point would still tend toward infinity at t=0 and thus occupy V(matter)=0 as you have written it. So this leaves a big question about what might be true of the Universe as a whole. Obviously unknown, but a range of theories abound, including some based on Inflation ideas.


*

*E= very big?


Surprisingly perhaps this need not be true. The reason is that gravitational energy (a kind of binding energy) counts negatively. So many theorists prefer the idea that E= small, with E=0 also being popular because then there is no net energy in the universe at t=0.
A: We can't know about what lies beyond the cosmological horizon. (roughly, any region more than 13.7 bn light years from us, though that's not completely right).  It is impoosible for any signal from there to travel to where we are.
What we can say is that all observations that we've made are consistent with an infinite universe everywhere filled with matter.  It turns out that this is very close to some very simple exact solutions of Einstein's equation.  So, it would make sense to take this model as our first guess/approximation as to what the universe, as a whole looks like.
