Is application of a Unitary operator equivalent to a phase change of a wave function? From my understanding, the application of a unitary operator does not change the physics of the system. It is just a more convenient representation in a new eigenbasis.
Now, I have also read today that the eigenvalues of a Unitary operator are complex numbers of moduli one. It is known that multiplication by a complex number is just a rotation in complex plane.
So, can the fact that unitary operator does not change the system be understood in the way that after the application of the unitary operator the eigenstate just changes a phase (rotates in the complex plane without scaling)?
Is this understanding faulty or incomplete or is this a correct way of looking at the application of the unitary operator?
 A: It is not equivalent for a general vector (state). You are correct that the eigenvalues of a unitary operator always have modulus one. But think about what that means. It means that if $|\psi \rangle$ is an eigenvector of a unitary operator U, then:
$$
U|\psi \rangle = e^{i\theta}|\psi \rangle
$$
So this is true for all eigenvectors, but not necessarily for a general vector. Consider, for example, a vector $|\psi\rangle = |\psi_1\rangle+|\psi_2\rangle$, where $|\psi_{1/2}\rangle$ are eigenvectors of $U$ with different eigenvalues $e^{i\theta_{1/2}}$. Then:
$$
U|\psi \rangle = U|\psi_1 \rangle+U|\psi_2 \rangle= e^{i\theta_1}|\psi_1 \rangle+e^{i\theta_2}|\psi_2 \rangle \neq e^{i\theta}|\psi \rangle
$$
So a unitary operator does preserve the magnitudes of vectors, by definition (so the probabilities are conserved - the "physics" is unchanged), but can still rotate the vector in your vector space.
A: There are a few concepts intertwined here that we can try to tease apart. Unitary transformations can indeed be considered changes of basis, unitary operators indeed cause changes of phase to their eigenstates, and multiplication by a complex number is indeed a rotation in the complex plane. However, lots of different things can happen when we put all of these concepts together, so let's work through a few ideas.
When does changing basis change the physics?
Consider the action of walking on a grid defined by two unit vectors $\mathbf{x}$ and $\mathbf{y}$. We can plot this on a graph, where the horizontal coordinate tells you how far you've moved in the $\mathbf{x}$ direction and the vertical coordinate tells you how far you've moved in the $\mathbf{y}$ direction. This sets the stage for different notions of changing basis.
If you are moving in the $\mathbf{x}$ direction and decide to apply a unitary operation to your velocity vector, all of a sudden you might find yourself moving in the $\mathbf{y}$ direction. The unitary has rotated your direction of walking, so now you are doing something physically distinct from what you were doing before.
If you are moving in the $\mathbf{x}$ direction and decide to rotate the graph while still moving horizontally, you have applied a unitary operation only on the underlying coordinate system and so you now seem to be moving in the $\mathbf{y}$ direction, which again has changed the physics.
If you are moving in the $\mathbf{x}$ direction and decide to rotate how you look at the graph, it will look $\mathbf{y}$ is the horizontal direction and $\mathbf{x}$ is the vertical direction, but it will look like you are moving in the vertical direction, so you are still moving in the $\mathbf{x}$ direction such that the underlying physics is unchanged! The distinction here is that we have applied the unitary to both the coordinate system and to the motion on the graph, which amounts to looking at the identical physical situation from a different perspective.
This distinction differentiates between when unitary operations do and do not change the physics. When a unitary operation changes the basis of everything, it is as if we are looking at the same physics from a different perspective, but when a unitary operation only changes the basis of the participants or of the coordinate system or of something else, the physics will indeed change.
Quantum description of unitaries changing basis vs. physics
Take a quantum state $|\psi\rangle$ that undergoes some transformation $|\psi\rangle\to A|\psi\rangle$. If we "change the basis" of the state $|\psi\rangle$ or the operator $A$, that means we are transforming them using a unitary $U$: $|\psi\rangle\to |\psi^\prime\rangle=U|\psi\rangle$ or $A\to A^\prime=U A U^\dagger$. Depending on the combinations of things that we change, the physics will or will not change.
If we only "change the basis" of $|\psi\rangle$, we see immediately that
$$A|\psi\rangle\neq A|\psi^\prime\rangle=AU|\psi\rangle.$$ Similarly, if we change the basis of $A$, we see that $$A|\psi\rangle\neq A^\prime|\psi\rangle=UAU^\dagger|\psi\rangle.$$ Thirdly, if we change the basis of both $|\psi\rangle$ and $A$, we still change the physics
$$A|\psi\rangle\neq A^\prime |\psi^\prime\rangle= UA U^\dagger U|\psi\rangle=UA|\psi\rangle!$$ The only way for us to not change the physics when we change the basis is if we also "look at the problem from a counter-rotated perspective" by changing the basis back at the end:
$$A|\psi\rangle=U^\dagger A^\prime |\psi^\prime\rangle.$$
What happens when we want to compare two things in quantum mechanics? We take an inner product $\langle\phi|\psi\rangle$. If we change the basis in which we compare the two things, nothing physically changes, because
$$\langle \phi|\psi\rangle=\langle\phi^\prime|\psi^\prime\rangle=\langle\phi|U^\dagger U|\psi\rangle.$$ The key is to change the basis for both things being compared. We can similarly see that
$$\langle \phi|A|\psi\rangle=\langle \phi^\prime|A^\prime|\psi^\prime\rangle,$$ but always have to remember that everything must have its basis changed for us to observe identical physics.
Unitaries only cause phase changes to their eigenstates
We can denote the eigenstates of our unitary $U$ by $|u\rangle$ and their eigenvalues by $\lambda_u=e^{i\theta_u}$. A system initially prepared in an eigenstate of $U$ will indeed observe no change in its physical properties after undergoing the unitary transformation:
$$U|u\rangle=e^{i\theta_u}|u\rangle,$$ and $e^{i\theta_u}|u\rangle$ only differs from $|u\rangle$ by a global phase.
All states can be formed from eigenstates of $U$, so does that mean that no state will change after undergoing the operation $U$? We write a pure state as
$$
|\psi\rangle=\sum_u \psi_u|u\rangle
$$ for some set of normalized coefficients $\{\psi_u\}$, which will evolve as
$$
U|\psi\rangle=\sum_u \psi_u e^{i\theta_u}|u\rangle.
$$ Now, if not all of the phases $\theta_u$ are identical, $|\psi\rangle$ has been changed by more than just a global phase! This means it will have physically different properties after undergoing the evolution. It is the fact that $|\psi\rangle$ is made from a superposition of different eigenstates $|u\rangle$ that gives rise to interesting dynamics.
We can extend this to the above considerations as well. A general operator $A$ can also be expressed in the basis comprised from the eigenstates $|u\rangle$:
$$A=\sum_{u_1,u_2}a_{u_1,u_2}|u_1\rangle\langle u_2|.$$ Changing the basis of $A$ changes the operator:
$$UAU^\dagger=\sum_{u_1,u_2}a_{u_1,u_2}e^{i(\theta_{u_1}-\theta_{u_2})}|u_1\rangle\langle u_2|.$$ This is a different operator from the original $A$ and so the unitary has changed the physics here.
Finally, if we take two states $|\psi\rangle$ and
$$|\phi\rangle=\sum_u\phi_u|u\rangle,$$ operate on them with $U$, and look at the various overlaps, we see that the physics is unchanged as before:
\begin{align}
\langle \phi^\prime|A^\prime|\psi^\prime\rangle&=\left(\sum_{u_3}\langle u_3|\phi_{u_3}^*e^{-i \theta_{u_3}}\right)
\left(\sum_{u_1,u_2}a_{u_1,u_2}e^{i(\theta_{u_1}-\theta_{u_2})}|u_1\rangle\langle u_2|\right)
\left(\sum_{u_4}\psi_{u_4}^*e^{i \theta_{u_4}}|u_4\rangle\right)
\\
&=\sum_{u_1,u_2,u_3,u_4}\phi_{u_3}^*e^{-i \theta_{u_3}}
a_{u_1,u_2}e^{i(\theta_{u_1}-\theta_{u_2})}
\psi_{u_4}^*e^{i \theta_{u_4}}\delta_{u_3,u_1}\delta_{u_2,u_4}\\
&=\sum_{u_1,u_2}\phi_{u_1}^*a_{u_1,u_2}
\psi_{u_2}=\langle \phi|A|\psi\rangle.
\end{align}
Again, changing the basis of everything leads to unchanged physics, but changing the basis of only one state, some states, one operator, or some other subset of everything will lead to different physics and different physical predictions.
