This is what observations tell us:
The galaxies do not change size because the gravitational attraction holding them together is much stronger than the expansion due to the "big bang etc"
To fit the expansion and many other observations, the current standard model of cosmology is the Big Bang model. The word "model" means we have found a mathematical representation which, with the fewest possible postulates( equivalent to axioms in mathematics) describes within errors a large number of observations.
History of the universe
In this plot, the balloon surface is an analogue of a transverse slice of the history, at a given time.
The Big Bang model has been evolving through the years, but the basic ingredients of special and general relativity are a backbone in the postulates, which is not changing.
The first naive model assumed that all matter/energy in the universe was at a singularity at the origin at (0,0,0,t=0), a four vector assigned to the point. New observations and development of theories, plus the necessity of consistency of gravitation with quantum mechanics, have modified this original "point" to a fuzzy quantum mechanical region, but still , all matter in the universe started from that region and is distancing itself as time goes on.
This is a model which fits observations and data, it is predictive, and the best we have up to now. It is at the frontier of research, but the rough lines will not change, because General Relativity is well validated, and also special relativity, i.e. the four dimensional form of space time will stay as is.
The model would not fit the data and observations ,if the center of the universe were only where we are. From gravitational lensing to spectroscopy from distanced sources to the cosmic microwave background radiation (why would it be uniform all through the skies if it is traveling away from us? there should be directionality) the current model fits data and observations .