Thanks a lot. This was helpful.

Sure. Totally understood.

Thanks. Essentially what I am asking is as to wether there are any ceveats to this. You are saying something very fundamental, the expectation value of the observable does not depend on the basis you measure it. I am wondering, are there any potential exceptions? For example, some people discuss a relaxation of unitarity sometimes.

Well, as discussed also by the answer below, I conclude that diagonalization is equivalent to transforming the original state as $|\psi \rangle \to U^{-1}|\psi \rangle$. So, as he says it is a change of basis. Naively this is equivalent to a (non-observableas always) gauge transformation. My questions essentially boils down to as there exist any ceveats to this.