What we directly observe is that the Universe was expanding and the expansion was accelerating during the recent five billions of years or so (it was actually not accelerating before that because the dark energy wasn't dominating). The prediction that it will continue to expand and accelerate results from a "clever scientific extrapolation" – from writing the most convincing equations that describe the past and from solving them in the future.
The past observations imply that the Universe is expanding because we see that distant galaxies were moving away from us in recent billions of years (because the light is redder – has a lower frequency – than it should have according to spectroscopy, a fact we attribute to the Doppler effect). If things are moving away from each other, their distance is increasing as you go from a smaller $t$ to a larger $t$ and this increase is called expansion, by definition. Your argument by which you want to confuse expansion and contraction looks utterly bizarre to me and I am pretty sure that I won't be the only one. Something that looks and behaves like a bomb after it detonates is called "expansion" and something that behaves like the rubber after we create a hole to a rubber balloon with a needle is called contraction or implosion or shrinking and most people know how to distinguish these two processes without asking questions on Physics Stack Exchange.
There's a clear difference between expansion and contraction, a difference that even most of the laymen and kids from the kindergarten probably appreciate. These two opposite processes lead to different observations and one may clearly say that the Universe has been expanding for quite some time.
In an analogous way, one may determine that the expansion was accelerating. This is found by reconstructing the acceleration rate that was valid "very recently" and the acceleration rate that was valid "billions of years ago": one needs to look at objects at very different distances to get the rates at at least two moments in this case. When the two rates are found, we see that the rate (=speed) of the expansion in recent years was faster than the rate (=speed) that applied billions of years ago. This increase of the rate (=speed) is what is called acceleration, by definition.