To extend @anna v's answer and link it to some concerns in your question:
Coherence in Superpositions
Coherences are the phase relationships between sections of our system.
As you noted, coherence is related to the superposition of states $|\psi\rangle = w_a |a\rangle + w_b e^{i\phi} |b\rangle$: if $\phi$ is well defined, then there is perfect coherence between $|a\rangle$ and $|b\rangle$. As Anna states, this proves to have crucial experimental consequences for interference of probabilities. If you add another state $|\psi'\rangle = e^{i\theta}( w'_a |a\rangle + w'_b e^{i\phi'} |b\rangle)$ in superposition with $|\psi\rangle$ to form $|\psi\rangle+|\psi'\rangle$, then it is clear the probabilities (to be measured in a particular basis state) depend on not only the relative phases within both $|\psi\rangle$ and $|\psi'\rangle$ but also the relative phase $\theta$ between them.
If the phase relationship is not there, the probability to be measured in a state adds classically, yielding $p_a=|w_a|^2+|w'_a|^2$ for example, with no phase dependence (i.e. interference) to be seen.
As Schlosshauer and Wikipedia either mention or allude, behaving classically (i.e. zero coherence between states) means we also lose quantum effects like quantum entanglement.
Coherences in the Density Matrix
The density matrix $\rho$ offers a nice way to smoothly transition between pure states with perfect coherence, and "mixed states" with less-than-perfect coherence between internal states. Any mixed state can be seen as a probabilistic sum of pure states (that by definition have perfect coherence).
If the magnitude of a coherence (an off-diagonal element $\rho_{ij}$ of the density matrix) is smaller than it would be for a pure state of the same populations (diagonal elements $\rho_{ii}$), then we know the state that $\rho$ represents is mixed and cannot fully be interfered. In model, this is because the phase information in the coherences have partially canceled in our probabilistic sum, leading to phase uncertainty.
If all of the coherences $\rho_{ij}$ are zero, then the mixed state is fully decohered and acts classically.
This justifies calling an off-diagonal element $\rho_{ij}$ a coherence, because its magnitude represents the strength of the phase relationship between the states, and the phase of one is precisely the "averaged" phase relationship of the mixtures.
