I think you are lacking a sort of assumption and mathematical base that build before quantum mechanics so that things should be rigorous and we don't do mistakes. So that's sort of summary I'm giving here. Jump to the next section if need no summary:
The two spaces that we need are $l_2$ and $L_2(a,b)$, the space of square-integrable function and space of square-integrable sequences defined for some real variable. These two spaces will play are a very important role in Quantum Mechanics as you know the state of the system is described by vectors in LVS and LVS will generally belong to these two.
Now What about the convergence of state in vector spaces? For a sequence of the complex number $\{z_n\}$ we want to ask if converge to some limit point, The sequences that do are what we call Cauchy sequences.
$$|z_n-z_m|\rightarrow 0 \ \ \ \ \ \mathrm{as} \ \ n,m\rightarrow \infty$$
$$\Rightarrow \lim_{n\rightarrow \infty}z_n \ \ \mathrm{exists}$$
This concept we carried to state vectors.
A set of point which encloses all it's limit points is called a closed set.
Now converting this concept to a vector space, We say, The sequence $\{|\psi_n\rangle\}$ in LVS is Cauchy sequence if $$\lim ||\psi_n-\psi_m||\rightarrow 0 \ \ \ \ \ \mathrm{as} \ \ n,m\rightarrow \infty $$
A linear vector space has a large number of vector and it's not necessary that every Cauchy sequence has a limit point which exists in the same space. If it does, then this space is called Complete LVS.
For finite-dimensional space, These concepts are not that useful but when talking of infinite-dimensional spaces you need these requirements.
A complete LVS with an inner product is called Hilbert space.
In quantum mechanics, We use Hilbert Space. A Hilbert space with a denumerable basis $\{\phi_n\}$ is called a separable Hilbert Space. In ordinary cases, We will work with this.
If this fact follows from linear algebra naturally why we need to interpret wavefunctions to be square-integrable in the first place?
The first postulate of Quantum Mechanics Says
The state of a system is described by state vector $|\Psi(t)\rangle$ in Hilbert Space.
The first postulate states that a particle is described by a ket $|\Psi(t)\rangle$ in a Hilbert
space which, you will recall, contains proper vectors normalizable to unity as well as
improper vectors, normalizable only to the Dirac delta functions.
From here it follows :
$$\int \psi^*(x)\psi(x)dx=1$$
But It might not always be the case. This is a good place to point out that the plane waves $e^{ipx/\hbar}$ all improper vectors, i.e., vectors that can't be normalized to unity but only to the Dirac delta function) are introduced into the formalism as purely mathematical entities. Our inability to normalize them to unity translates into our inability to associate with them a sensible absolute probability distribution, so essential to the physical interpretation
of the wave function. In the present case, we have a particle whose relative
probability density is uniform in all of space. Thus the absolute probability of finding
it in any finite volume, even as big as our solar system, is zero. Since any particle
that we are likely to be interested in will definitely be known to exist in some finite
volume of such large dimensions, it is clear that no physically interesting state will
be given by a plane wave.
Secondly why is it the probability, what's the harm in calling it just a random scalar?
This is what the content of the 4th postulate is. So If You call it a scalar without any physical interpretation for it that helps you to describe the system then there is no use for it. There is reason to call it probability (Ask a different question). No one does a length calculation to find a number that has no physical significance.