Not all self-adjoint operators are observables? The WP article on the density matrix has this remark:

It is now generally accepted that the description of quantum mechanics in which all self-adjoint operators represent observables is untenable.[17][18]

The first footnote is to the appendix in Mackey, Mathematical Foundations of Quantum Mechanics. I have a copy of Mackey, but I am unable to connect the material in the appendix to this statement. I also don't know anything about C* algebras -- and Mackey doesn't seem to mention them either.
Can anyone explain this at an elementary level, maybe with a concrete example of a hermitian operator that wouldn't qualify as an observable?
I assume this issue only arises in infinite-dimensional spaces...? Intuitively, I don't see how it could be an issue in a finite-dimensional space.
 A: The appendix of Mackey talks about superselection rules, and indeed superselection is the phenomenon where there are self-adjoint operators that are not observables. Whether this is obvious or not depends on how one defines "superselection".
The standard definition would be that the Hilbert space $H$ splits into the direct sum $H_1\oplus H_2$ such that for all $\lvert\psi\rangle\in H_1, \lvert \phi\rangle\in H_2$ and all observables $A$ we have that $\langle \psi \vert A\vert \phi \rangle = 0$, which implies that $AH_1 \subset H_1,AH_2\subset H_2$ for all observables, but which is clearly not true for all self-adjoint operators on $H_1\oplus H_2$.
An easy (albeit somewhat artificial) example of a superselected system is when we take $H_1$ to be the state space of a boson and $H_2$ the state space of a fermion, see also this answer of mine. Other examples can arise from theories with spontaneous symmetry breaking where states belonging to different VEVs cannot interact with each other and form superselection sectors.
A: In general, for physical reasons not all self-adjoint operators are observables. I will discuss the problem at the level of von Neumann algebras that are (perhaps unawares) more familiar to physicists. There are two further possibilities leading to a bit differently articulated answers: $C^*$-algebras and lattice of elementary propositions theory.  
Let us start from observing that self-adjoint operators of a quantum system $S$ are not enough to exhaust the set of operators useful in a description of  $S$ itself. For instance, if $A$ is an observable $U_a = e^{-iaA}$, where $a\in \mathbb R$, defines a group of continuous symmetries associated to that observable and this operator (for fixed $a \in \mathbb R$) is not self-adjoint. 
The class of useful operators is constructed this way. First notice that the observables can always be reduced to a larger class of bounded operators: if $A$ is unbounded (i.e. the set of attained values defining its spectrum $\sigma(A)$ is an unbounded subset of $\mathbb R$) the class $\{A_n\}_{n \in \mathbb N}$ of bounded self-adjoint operators 
$$A_n = \int_{(-n,n] \cap \sigma(A)} a dP^{(A)}(a)\tag{1}$$
where $P^{(A)}$ is the spectral measure of $A$, encompasses the whole information of $A$ itself. In particular, if $\psi \in D(A)$ then
$$A\psi = \lim_{n\to +\infty}A_n \psi\tag{2}$$ and
$$\cup_{n \in \mathbb R} \sigma(A_n) = \sigma(A) \:\:\: \mbox{up to the possible element $0 \in \sigma(A)$}$$
(2) says that the class of bounded operators $A_n$ constructs $A$ in the strong operator topology.
This way every observable is de-constructed into a set of bounded observables. Next one considers products of all possible complex combinations of bounded observables including infinite combinations in the strong-operator topology.
The obtained set of operators $R_S$ includes all conceivable operators useful for a quantum system extracted from observables (including the bounded  observables themselves coinciding with the self adjoint elements of $R_S$,  their spectral measures and the continuous symmetries generated from them). 
This class of operators $R_S$ is known as the von Neumann algebra (or $W^*$ algebra) of a given quantum system $S$.
The natural question, which in complex Hilbert spaces is equivalent to your initial question reads as follows.
For a quantum system described in the Hilbert space $H$, does $R_S = B(H)$?
where $B(H)$ is the largest von Neumann algebra in $H$ consisting of all bounded operators $A : H \to H$.
(Indeed, if all self-adjoint operators are observables, then $R_S$ includes all bounded self-adjoint operators and their complex combinations and thus it includes the full $B(H)$ because every bounded operator is a complex linear combination of a couple of bounded self-adjoint operators. If, vice versa, $R_S = B(H)$, then every self adjoint operator is an observable since, by definition, all self-adjoint operators in $R_S$ are the observables of $S$.)
Physics decides actually.
There are two independent possibilities when $R_S \subsetneq B(H)$.
$\:\:\:\:$(A) Presence of Abelian superselection rules. 
This means that 
there are orthogonal projectors $P_k$ in $R_S$ such that 
(i) $P_k$ commutes with every element of $R_S$,
(ii) $P_k \perp P_h$ if $k \neq h$,
(iii) $\oplus_k P_k =I$. 
The projection spaces $H_k = P_k(H)$ are called superselection sectors and $H$ is the orthogonal sum of them $H= \oplus_k H_k$ due to (ii) and (iii).
In this case, evidently $R_S \subsetneq B(H)$ because, barring trivial cases,  $B(H)$ includes some operators which do not commute with some $P_k$. 
On the physical side this means that for instance there is no way to distinguish a vector state $$\psi = \sum_k c_k\psi_k\tag{1}$$ where $\psi_k \in H_k$ are unit vectors, and the mixture $$\rho_\psi = \sum_k |c_k|^2|\psi_k\rangle \langle \psi_k|$$
Using the fact that $A^\dagger=A \in R_S$ commutes with every $P_k$ it is easily proved, for instance, that
$$tr(\rho_\psi A) = \langle \psi|A\psi \rangle$$
but the result extends to probabilities of outcomes and so on. Another way to physically illustrate this phenomenon is saying that 
no coherent superposition of pure states of different superselection sectors $H_k$ is possible
As a typical example, consider the observable electric charge $Q$ for an electrically charged quantum system (also a quantum field). $Q$ has a discrete, in general unbounded,  spectrum and the superselection rule of electric charge states that all observables commute with it. It immediately implies that the eigenspaces $H_q$ of $Q$ are superselection sectors and that an Abelian superselection rule takes place. All that is equivalent to stating that no coherent superposition of states with different charge is allowed.
Regarding the initial question, consider for instance two different values of the charge $q$ and $q'$ and associated eigenvectors $|q\rangle$ and $|q'\rangle$. The self-adjoint operator
$$A = |q\rangle\langle q'| +  |q'\rangle\langle q|$$
 does not  define an observable  because it does not commute with the projector $P_q$ onto $H_q$.
Similar well-known superselection rules in quantum physics are the superselection rule of mass (Bargmann's superselection rule) for non-relativistic quantum systems and the superselection rule of angular momentum (integer values vs semi-integer values).
$\:\:\:\:$(B) The theory admits a (non-Abelian) gauge group.
This means that there is a class of observables $R'_S$, generally called the commutant of $R_S$, whose elements commute with every element of $R_S$ but some elements of the commutant are not included in $R_S$ itself, as it happens for (A) instead. This is in particular the case if the commutant $R_S'$ includes  a pair of non-commuting operators at least. 
Let us stick to this case (which is the only possible actually due to the "double commutant theorem"). It is easy to prove that the unitary elements of $R_S'$ give rise to a non-Abelian unitary group. In other words all observables of $R_S$ must commute with unitary operators forming a non-abelian group, the (non-Abelian) gauge group of the theory. 
An example consists of the description of quarks: all their observables must commute with a unitary representation of $SU(3)$ (color).
As an example of self-adjoint operators which are not observables in this case, just consider the self-adjoint generators of the relevant $SU(3)$ representation: they cannot commute with the representation (unless trivial) so they cannot be interpreted as quark observables because all observables commute with the representation.
(I discussed these topics in my 2013 Springer book on "Spectral theory and QM" and in the next very enlarged 2018 edition in print which includes an even wider discussion.)
ADDENDUM. It should be clear that in both cases where not all self-adjoint operators are observables the one-to-one correspondence
pure states $\leftrightarrow$ unit vectors up to phases
dramatically fails. 
An example was provided in the case of Abelian superselection rules. Notice that, referring to that example $\psi$, $\rho_\psi$  but also (comparing with (1))
$$\psi' = \sum_k e^{i\theta_k}c_k \psi_k$$
are physically indistinguishable. In the case of presence of a gauge group $\psi$ and $U\psi$, where $U$ is a unitary operator of the gauge group, determine the same state.
Another physically important consequence of the presence of a gauge group  is that no maximal set of commuting observables may exist. The proof is a bit technical and I omit it. In practice, we cannot prepare the quantum system in a preferred state by a selective sequence of measurements of compatible observables (with pure point spectrum).
A: A self adjoint operator is an observable if you con build a device to measure
its eigenvalues. Each time a system is prepared in a given eigeinvector your
device will give the same result.
You can do it locally if you are near the system or far from it. 
Problems appear when you are in a curved space time. There may be an horizon
between you and the system. As you have no access to all the degrees of freedom you have to trace out on the far dofs . 
a pure eigenvector will appear as a mixture and will not give always the same result.
An occupation number is a self adjoint operator.
If the prepared system is the vacuum of Minkowski's
space time an eternally accelerated observer will see it as a thermal bath
at a given temperature. for him the measured operation number is never zero.
And of course someone will never be able to measure the occupation number of fermions with a given spin at a place behind the horizon of a BH.
