What is the significance of a Hilbert space? I see the term Hilbert space thrown around a lot. At first I thought it is just when you have complex vectors and define an inner product between them. However, it seems to be a lot more than that because the term is used over and over again. I associate it with people talking about having an "orthonormal basis" and decomposing the wave function into "modes" and I don't understand what this means and why you would do it. How does it help you solve a physics problem?
 A: I would like to add to alephzero's good answer a very important characteristic of a Hilbert-space -- without it the Hilbert-space would loose enormously of its significance --- which is completeness.
Let's take the example of Fourier-series. For instance a function alternating uncontinously between the only values $\pm1$can be represented by an infinite series of continuous functions. Abstractly we would write the function like
$$f(x) \approx g_N(x)=\sum_{m=0}^N a_m e_m(x)$$
where $e_m(x)$ are the Fourier-modes which are continuous trigonometrical functions. Knowing $f(x)$ we can compute the amplitudes of the Fourier-modes that contribute to the decomposition as
$$ a_m =\langle f(x),e_m(x)\rangle$$
where $\langle u,v\rangle $ is the inner product defined for the Hilbert-space. We actually have the (almost) the same structure of finite-dimensional vector-space $V$ where we can decompose a vector $v\in V$ like
$$ v = \sum_{m=0}^N \langle v,e_m \rangle e_m$$
In Fourier theory the same is true apart from the "little detail" that the decomposition requires an infinite number of Fourier-modes or (in language of linear algebra) base vectors
$$\lim_{N\rightarrow\infty} \sum_{m=0}^N a_m e_m(x) = f(x)$$
with $a_m = \langle f(x),e_m(x)\rangle $. However, this is by far not trivial.
I demonstrate this by an example. We could define on the vector space of continuous functions $C^0(\mathbb{R})$ a scalar product $\langle f,g\rangle = \int f(x)g(x)dx$
and "technically" we would have all what we need. Almost,  because there is a problem. Upon taking the limes $N\rightarrow \infty$ the result of the sum does not converge simply because the function $f(x)$ not continuous (Note that as long as $N$ is finite a sum of continuous Fourier-modes is continuous, i.e. $\in C^0(\mathbb{R})$, but the final result at $N\rightarrow\infty$ is not)
Therefore the vector space of continuous functions with an integral scalar product is not a Hilbert-space.
Of course there is a fix to this problem: An appropriate function space has to be chosen so that the limes of $N\rightarrow \infty$ for the Fourier-mode decomposition exists. In this context it would be the function space of square integrable functions $L^2(\mathbb{R})$. Actually physicists almost never care in which function space a infinite series converges, for them it only a concern of mathematicians. But one should keep in mind that the completeness of a Hilbert-space guarantees that infinite series or solutions of differential equation actually exist (if the adequate function space is chosen for them to exist).
And as a last remark. Hilbert-spaces are ubiquitous. In particular when solving differential equations, the solutions can almost never be written as an elementary function to be found in "mathematical dictionary". In many cases the solution is constructed in a Hilbert-space whose elements is decomposed in an appropriate series of infinite (often indeed orthonormal) base-modes (Hermite, Legendre,  Bessel etc.) which makes the concept so important.
A: Hilbert spaces are complex linear (metric) spaces that are generalised from the finite-dimensional case in one particularly subtle feature --among others: they must be closed also for infinite sums. This amounts to saying that the limit of infinite sequences of states must also be a valid state.
These expansions are most useful when they are done with respect to a particular observable (a property of the system which is possible to measure).
Thus, in an expansion such as,
$$
\left|\psi\right\rangle =c_{1}\left|1\right\rangle +c_{2}\left|2\right\rangle +\cdots
$$
$$
=\sum_{a}c_{a}\left|a\right\rangle 
$$
Every one of the $\left|a\right\rangle $'s is distinguished from each other by having a different expected value for the observable $A$. In the formalism of quantum mechanics this is phrased as "the probability that upon measuring observable $A$ in state $\left|\psi\right\rangle $ the result is $a$ corresponds to $\left|\left\langle \left.a\right|\psi\right\rangle \right|^{2}$.
Entities of the form $\left| \psi \right\rangle $ represent states, and binary expressions of the form $\left\langle \left.a\right|\psi\right\rangle $ represent overlaps of different states, i.e., how much of the state with definite value $a$ participates in the state $\left| \psi \right\rangle $.
A: A Hilbert space is a mathematical object which has properties analogous to the "common sense" properties of 3-dimensional Euclidean space. The objects correspond to "geometrical points," there is a definition of "distance," and there are concepts of "continuity" and "limits".
So all the high-school-level math you learned in an informal way, thinking about "common-sense" geometry in two or three dimensions (Pythagoras' theorem, the parallelogram law for vectors, calculus, etc) also "works" in any Hilbert space.
The most important consequence is that all this math still "works" when the Hilbert space is analogous to an infinite-dimensional Euclidean space.
To learn about concepts like an "orthonormal basis", and "decomposing a function into modes". you need to study linear algebra - in particular, eigenvalues and eigenvectors. This is a generalization of "high-school physics" ideas like the fact that a stretched string, or the air in a pipe, can vibrate at different resonant frequencies (e.g. its "fundamental frequency" plus "harmonics"). The eigenvalues are analogous to the frequencies, and each eigenvalue has a corresponding eigenvector or "mode" corresponding to the shape of the vibration.
In the general case, the motion of the string is a combination of all the harmonics. Some important properties can be calculated separately for each harmonic and then summed. For example the total energy in the vibrating string is the sum of the energy in each individual harmonic - there are no "extra energy terms" coming from some sort of "interaction between two different harmonics."
