The "changing frames" language might be confusing the issue. Coordinate systems and observers are two independent concepts. All of the observers in any given scenario can be described using a single coordinate system. The choice of coordinate system doesn't matter at all, aside from mathematical convenience. Some coordinate systems might be more convenient for some observers, but that's the only connection between them. It's a matter of convenience, not necessity.
So, the answer depends on what you mean by "changing frames":
If it means changing coordinates, so that the given unitary transformation is being used to represent a mere change of coordinates, then we need to apply it to everything (operators and states) so that it doesn't change any predictions. Changing the coordinate system can't have any physical consequences, because coordinates are just labels.
If it means changing the observer, then the answer depends on what kind of model we're using. I'll call them in-practice models and in-principle models.
In practice, solving the Schrödinger equation for a one multi-electron atom is already difficult enough, nevermind solving the Schrödinger equation for a person made of jillions of molecules. So, instead of treating the observer as a physical entity, we usually omit the observer, and instead we artificially specify which observable is being measured. That's what I mean by an in-practice model. If $O$ is an observable and $U$ is a unitary operator that implements a rotation, then $U^\dagger OU$ is a different observable. In any given state, we could choose to measure either $O$ or $U^\dagger OU$, and these should typically give different results. So in this case, we should apply $U$ only to the observable, not to the state. Equivalently, we could apply $U$ only to the state, not the observable. These two options are equivalent, just like the Schrödinger and Heisenberg pictures are equivalent. (This equivalence comes from the fact that applying the same $U$ to both the observable and the state doesn't have any physical consequences: it's just a change of basis.)
In principle, quantum theory allows using a model that includes complex macroscopic observers (like people) as part of the physical quantum system. That's what I mean by an in-principle model. In this type of model, changing the observer means changing the state, because the state describes the whole physical system including the observer. But this isn't a symmetry operation. If a person moves to a different location in the laboratory, then the state — which describes both the laboratory and the person — has changed in some very complicated way. Such a complicated change cannot be described by anything as simple as an overall translation or rotation. We can still describe it as a unitary transformation if we want to, because any change from one given state to another given state can be described by a unitary transformation, but it's not a symmetry.
By the way, even with an in-principle model, we still need to tell the theory which outcome we actually experience when an observable is measured. Using an in-principle model doesn't avoid the need for the usual projective-measurement rules, but it does allow us to defer application of those rules until after the physical process of observation — which the model itself describes — is complete.