Regarding the meaning of "universe":
Unqualified, the Universe is all there is, was, and will be. In other words, there is nothing that is outside of, independent of, stands apart from, the Universe.
The Universe is commonly defined as the totality of existence
However, it is often the case that "universe" is short for "observable universe":
The observable universe consists of the galaxies and other matter that
can, in principle, be observed from Earth in the present day because
light (or other signals) from those objects has had time to reach the
Earth since the beginning of the cosmological expansion.
Both popular and professional research articles in cosmology often use the
term "universe" to mean "observable universe". This can be justified
on the grounds that we can never know anything by direct
experimentation about any part of the universe that is causally
disconnected from us, although many credible theories require a total
universe much larger than the observable universe.
Regarding the Big Bang, you ask: so if the universe was initially a point then what was outside that point?
This is a common question and the answer takes some getting used to. The metric expansion of space does not require that there be something 'outside' within which space is expanding.
Regardless of the overall shape of the universe, the question of what
the universe is expanding into is one which does not require an answer
according to the theories which describe the expansion; the way we
define space in our universe in no way requires additional exterior
space into which it can expand since an expansion of an infinite
expanse can happen without changing the infinite extent of the
expanse. All that is certain is that the manifold of space in which we
live simply has the property that the distances between objects are
getting larger as time goes on. This only implies the simple
observational consequences associated with the metric expansion
explored below. No "outside" or embedding in hyperspace is required
for an expansion to occur. The visualizations often seen of the
universe growing as a bubble into nothingness are misleading in that
respect. There is no reason to believe there is anything "outside" of
the expanding universe into which the universe expands.