What exactly implies the need of quantum mechanics for self-adjoint and not only symmetric operators? We know that quantum mechanics requires self-adjoint operators, not only symmetric. Can we say that this follows ONLY from the two following axioms of quantum mechanics, namely that 


*

*each observable $a$ corresponds to a linear operator $A$ 


and 


*an expectation value of a measurement of $a$ must be real


?
I thought these two imply only the need for a Hermitian (i.e. symmetric) operators (because a linear Hermitian operator has real eigenvalues)  and that the need for self-adjointness was somehow connected to an additional requirement such as unitarity of the time-evolution operator. What is the missing piece? 
(I know how the two terms are defined, e.g. here.)
 A: In QM, a real valued observable $A$ is mathematically represented by a projector valued measure over $\mathbb{R}$, $P^{(A)}$, i.e., if $E$ is a Borel subset of the real line, then $P^{(A)}(E)$ is a projector representing the proposition "the outcome of measuring $A$ falls in $E$". In principle, that's all you need for representing, mathematically, observables in QM (I'm assuming the fundamental formulation of the theory via the non-distributive lattice of propositions).
But, by the Spectral Theorem for unbounded self-adjoint operators (proved by von Neumann), we know that given an observable $A$ represented by the projector valued measure $P^{(A)}$, there's a self-adjoint operator, also called $A$, such that the following decomposition is unique
$$A=\int_{\mathbb{R}}\lambda\,\mathrm{d}P^{(A)}(\lambda)$$.
The spectrum $\sigma(A)\subset\mathbb{R}$ coincides with the support of $P^{(A)}$.
This is how and why we get the usual one to one correspondence between observables and self-adjoint operators in QM.
A: Unitarity of the time-evolution operator is exactly the point:
Stone's theorem (see e.g. Reed, Simon: Theorems VIII.7, VIII.8) tells us


*

*If $A$ is self-adjoint, the spectral theorem holds.
This gives us a functional calculus which makes it possible to define $U(t) = e^{itA}$ in the first place.

*A such defined $U(t)$ is a strongly continuous unitary group.

*If $U(t)$ is a strongly continuous unitary group, then there exists a self-adjoint $A$ such that $U(t) = e^{itA}$.


Edit: This only tells us why the Hamiltonian should be self-adjoint. QuantumLattice's answer is better.
