What exactly is a bound state and why does it have negative energy? Could you give me an idea of what bound states mean and what is their importance in quantum-mechanics problems with a potential (e.g. a potential described by a delta function)?
Why, when a stable bound state exists, the energies of the related stationary wavefunctions are negative?
I figured it out, mathematically (for instance in the case of a potential described by a Delta function), but what is the physical meaning?
 A: Barry Simon writes:

One of the more intriguing questions concerns the
presence of discrete eigenvalues of positive energy (that is, square-integrable
eigenfunctions with positive eigenvalues) . There is a highly non-rigorous but
physically appealing argument which assures us that such positive energy “bound
states” cannot exist. On the other hand, there is an
ancient, explicit example due to von Neumann and Wigner which presents
a fairly reasonable potential $V$, with $V(r) \to 0$ as $r \to 
\infty$ and which possesses an
eigenfunction with $E = 1$.
The potential$$V(r)=\frac{-32 \sin r[g(r)^3 \cos r-3g(r)^2\sin^3r+g(r)\cos r+sin^3r]}{[1+g(r)^2]^2}$$
with $g(r)=2r-\sin2r$ has the eigenvalue
+1 with eigenfunction
$$u(r)=\frac{\sin r}{r(1+g(r)^2)}$$
On Positive Eigenvalues of One-Body
Schrodinger Operators

In a 2019 paper Simon explains:

Consider on $R^ν$ , the equation $(−∆ + V )φ = λφ$ with $V (x) \to 0$ as $|x| \to \infty$ and
$λ > 0$. Naively, one might expect that no solution, $φ$, can be in $L^2(R^ν , d^νx)$. The
intuition is clear: classically, if the particle is in the region $\{ x \mid |x| > R\}$, where $R$ is
picked so large that $|x| \gt R \Longrightarrow V (x) \lt λ/2$ and if the velocity is pointing outwards,
the particle is not captured and so not bound. Due to tunneling, in quantum theory,
a particle will always reach this region so there should not be positive energy bound
states. This intuition of no embedded eigenvalues is incomplete due to the fact that
bumps can cause reflections even when the bumps are smaller than the energy,
so an infinite number of small bumps which do not decay too rapidly might be
able to trap a particle.

A: Mathematically, bound states are states that decay sufficiently fast at infinity, so that the probability of finding the particle they describe in far away regions of space is negligible.
It has long been conjectured, based on physical intuition, it is the case for the meaningful quantum mechanical states, such as the eigenfunctions of the Hamiltonian (it is not expected that an atomic electron has a sensible probability of being at infinite distance from its nucleus).
This has been proved mathematically in the eighties, mainly by S.Agmon. Roughly speaking, the result is the following: eigenfunctions of the Schrödinger operator (i.e. corresponding to the discrete spectrum) are exponentially decaying in space. So if $\psi_n(x)$ are such eigenfunctions, $\lvert \psi_n(x)\rvert\leq A e^{-B\lvert x\rvert}$, for some positive constants $A,B$. 
A: If you have a copy of Griffiths, he has a nice discussion of this in the delta function potential section. In summary, if the energy is less than the potential at $-\infty$ and $+\infty$, then it is a bound state, and the spectrum will be discrete:
$$
\Psi\left(x,t\right) = \sum_n c_n \Psi_n\left(x,t\right).
$$
Otherwise (if the energy is greater than the potential at $-\infty$ or $+\infty$), it is a scattering state, and the spectrum will be continuous:
$$
\Psi\left(x,t\right) = \int dk \ c\left(k\right) \Psi_k\left(x,t\right).
$$
For a potential like the infinite square well or harmonic oscillator, the potential goes to $+\infty$ at $\pm \infty$, so there are only bound states.
For a free particle ($V=0$), the energy can never be less than the potential anywhere***, so there are only scattering states.
For the hydrogen atom, $V\left(r\right) = - a / r$ with $a > 0$, so there are bound states for $E < 0$ and scattering states for $E>0$.

Update
*** @Alex asked a couple questions in the comments about why $E>0$ for a free particle, so I thought I'd expand on this point.
If you rearrange the time independent Schrödinger equation as
$$
\psi''= \frac{2m}{\hbar^2} \left(V-E\right) \psi
$$
you see that $\psi''$ and $\psi$ would have the same sign for all $x$ if $E < V_{min}$, and $\psi$ would not be normalizable (can't go to $0$ at $\pm\infty$).
But why do we discount the $E<V_{min}=0$ solutions for this reason, yet keep the $E>0$ solutions, $\psi = e^{ikx}$, when they too aren't normalizable?
The answer is to consider the normalization of the total wave function at $t=0$, using the fact that if a wave function is normalized at $t=0$, it will stay normalized for all time (see argument starting at equation 147 here):
$$
\left<\Psi | \Psi\right> = \int dx \ \Psi^*\left(x,0\right) \Psi\left(x,0\right) = \int dk' \int dk \ c^*\left(k'\right) c\left(k\right) \left[\int dx \ \psi^*_{k'}\left(x\right) \psi_k\left(x\right)\right]
$$
For $E>0$, $\psi_k\left(x\right) = e^{ikx}$ where $k^2 = 2 m E / \hbar^2$, and the $x$ integral in square brackets is $2\pi\delta\left(k-k'\right)$, so
$$
\left<\Psi | \Psi\right> = 2\pi \int dk \ \left|c\left(k\right)\right|^2
$$
which can equal $1$ for a suitable choice of $c\left(k\right)$.
For $E<0$, $\psi_k\left(x\right) = e^{kx}$ where $k^2 = - 2 m E / \hbar^2$, and the $x$ integral in square brackets diverges, so $\left<\Psi | \Psi\right>$ cannot equal $1$.
A: It means the same thing it means in classical mechanics: if it is energetically forbidden to separate to arbitrarily large distance they are "bound".
The Earth is gravitationally bound to the Sun and the Moon to the Earth. Electrons in a neutral atom are electomagnetically bound to the nucleus. A pea rolling around in the bottom of a bowl is bound.
By contrast the Voyager probes are (barely) unbound and will fly (slowly) off into the galaxy.
A: *

*Cutting through the technicalities Wikipedia essentially defines a bound state in $d$ spatial dimensions as

*

*(i) a solution $\psi$ to the TISE
such that

*

*(ii) $\psi\in{\cal L}^2(\mathbb{R}^d)$
is square integrable/normalizable.



*Condition (ii) is (under relatively mild assumptions$^1$)
equivalent to that

*

*(iii) $\lim_{R\to\infty}\int_{\mathbb{R}^d\backslash B({\bf 0},R)} d^dr~|\psi({\bf r})|^2 ~=~0,$
where $B({\bf 0},R)$ denotes a ball at the origin with radius $R$.


*Sufficient conditions for a bound state are condition (i) together with

*

*(iv) if the potential is asymptotically bigger than the energy, in the sense that
$$\exists k,K>0\forall |{\bf r}|\geq K:~~ V({\bf r})-E  ~\geq~ \frac{\hbar^2k^2}{2m},$$
and

*

*(v) if $\psi$ is bounded
$$\exists c>0\forall {\bf r}\in\mathbb{R}^d:~~ |\psi({\bf r})|~\leq ~c.$$
See e.g. my Phys.SE answer here for the 1D case.


*Condition (iv) hints that there is a threshold energy (given by the infimum asymptotic value of the potential) above which a continuum of scattering states exist.


*Above the threshold energy, the solution (i) is oscillatory and on physically grounds generically expected to violate condition (ii). This explains the second half of OP's title question.


*However, it should be stressed that in special cases bound states in the continuum may exist, see e.g. the answer by Keith McClary.
--
$^1$ Here we assume for simplicity that the potential $V$ is not so singular in the interior/bulk, that the solution (i) becomes singular there.
A: 
Why, when a stable bound state exists, the energies of the related
  stationary wavefunctions are negative?

First, we need to understand that Energy is relative, both in Quantum and in Newtonian mechanics. This means that different reference frames will determine different values for the energy of a given object at a given point in space and time. 
In this case, we are allowing to have negative Energy values for delta potential energy by the following.

()=− () (where  is a positive number)

This means, in this reference frame we consider all Energy values with minimum () value that is negative infinity. Therefore, we are allowed to solve the Schrodinger Equation for all the negative and positive values of Energy.
In contrast, for the free particle problem's reference frame, we consider minimum () value is zero and all the Energy values should be positive for well-defined solutions.
However, the conservation of energy holds in any reference frame. This means that, in a single reference frame the total energy (KE + PE) will be constant over time.
