I've been reading papers about nanomechanical oscillators, and the concept of quality factor often pops up. I understand to some extent about Q factor in classical sense, but since nanomechanic oscillators are often treated quantumly, what does Q factor mean then?
There is no difference in interpretation. In both classical and quantum oscillators, if they have any dampening, the Q factor is higher the lower the dampening is. In quantum mechanics, it is common to relate the dampening to the half-life, but as far as I can tell, there is no further difference.
EDIT: the definition of Q is still the same, $\Delta f / f$ (though now that I re-read the wiki link you gave, I see that this is true only for high Q). If your system is treated quantumly, you just need to calculate the decay of the state you are interested in, or the linewidth of the energy level.
This is just a quick answer that I'm hoping will be superseded, because I'd like to see a good answer to this question. In the context of microwave resonators etched on superconducting chips, I know from experience that the Q factor seems to be basically the $T_1$ time. Sorry I can't be more help -- this is a point of confusion for me too.
If you have a hydrogen atom in empty space in the a p-state and ask about the transition to the ground state, Quantum Mechanics tells you that the coupling between the states is given by something called Fermi's Golden Rule. There is also something called the Einstein A coefficient which gives the probability per unit time of spontaneous emission. Since we said the atom is in empty space, that is the only transition possible, so it all somes to the same thing.
Since there is a frequency of the transition, and since there is a total elapsed time (to a power of 1/e), you can take the product of these which is basically the number of oscillations during the transition. That's the quality or Q-factor.
This all makes perfect sense in a very classical way. If you look at the formula for calculating the coupling between the states, you will see that there is something which basically comes to the dipole moment of the superposition of the s and p states. In fact, when those states are in superposition, they represent an oscillating dipole. This is well known and you can see it demonstrated in many applets such as this one by Paul Falstad The thing not everyone knows is that if you interpret this wave function as charge density, then it has electromagnetic properties that are readily calculated by Maxwell's equations. One such property is the rate at which power is being radiated classically. If we divide this rate into the total energy of the state, we get a characteristic lifetime.
It should come as a surprise to nobody that this is the same characteristic lifetime given by the Einstein A coefficient. The Einstein B coefficients, and of course the Q factors, are similarly calculable. In otherwords, the radiative properties of Schroedinger's hydrogen atom are given exactly by applying Maxwell's equations. There is no need to assume that energy is absorbed or given off in discrete lumps.