Two Level Atom Rotating Frame Introduction
I am trying to work out the Rabi problem. In particular I am trying to work it out from a perspective focusing on the operators rather than the kets. Think Heisenberg picture rather than Schrodinger picture. It's a bit tricky because in either the ket-centric or operator-centric view you must go into the "rotating frame" which is an interaction frame.
This question is very long and involves a section where I explain my understanding of unitary transformations and switching reference frames just so responders know what my understanding is. The main question I'm asking is at the very bottom.
Short Synopsis of Question
I am trying to derive the Hamiltonian shown in the Wikipedia page for the Autler-Townes effect or AC stark shift.
$$
H = -\Delta \lvert e \rangle \langle e \rvert + \frac{\Omega}{2}\left(\lvert e \rangle \langle g \rvert + \lvert g \rangle \langle e \rvert\right)
$$
I am beginning with the Schrodinger picture Hamiltonian for a two-level system coupled to an oscillating driving field
$$
H = \omega_0\lvert e \rangle \langle e \rvert + \frac{\Omega}{2}\left(e^{-i\omega t} \lvert e \rangle \langle g \rvert + e^{+i\omega t}\lvert g \rangle \langle e \rvert\right)
$$
I want to derive the wikipedia Hamiltonian specifically by performing a unitary transformation to get into the rotating frame and simply applying the formulas for effective interaction picture Hamiltonians. In particular, I don't want to have to derive equations of motion and then "notice" what the effective Hamiltonian should be. I'd also like to work in the Heisenberg/operator picture rather than Schrodinger picture as is often done for this problem.
I'm having a hard time doing this because it seems when you change which "picture" you are working in you only add or subtract terms from the Hamiltonian. Thus it doesn't really seem possible to remove the explicit time dependence coming from the exponentials from the Hamiltonian. I think there must be an additional trick that I need to play..
Can anyone provide any tips for me to derive this time-independent rotating frame Hamiltonian?
Setup
I'll label Heisenberg kets and operators with an $S$ subscript, Heisenberg with $H$, and I will use two more subscripts $N$ and $F$ for two other "frames" I might bring up. $\hbar = 1$ whether I remember to omit it or not...
Set up:
\begin{align}
H_S &= \omega_0 \frac{1}{2}(1+ \sigma_{z,S}) + \frac{\Omega}{2}\left(e^{+i\omega t} \sigma^-_S + e^{-i\omega t}\sigma^+_S \right)\\
&=\omega_0 \left(|e\rangle \langle e|\right)_S + \frac{\Omega}{2}\left(e^{+i\omega t} \sigma^-_S + e^{-i\omega t}\sigma^+_S \right)
\end{align}
Note that I have already made the rotating wave approximation. I don't have problems with the rotating wave approximation at the moment. Notice that there is the atomic frequency $\omega_0$ and the drive frequency $\omega$. I define $\Delta = \omega - \omega_0$. We have
\begin{align}
\sigma^+_S &= (|e \rangle \langle g|)_S\\
\sigma^-_S &= (|g \rangle \langle e|)_S\\
\sigma_{z,S} &= (|e \rangle \langle e|)_S - (|g\rangle \langle g|)_S\\
1 &= (|e \rangle \langle e|)_S + (|g\rangle \langle g|)_S\\
\frac{1}{2}(1+\sigma_{z,S}) &= (|e \rangle \langle e|)_S
\end{align}
In this picture none of the operators have any time dependence (except $H_s$ which has explicit time dependence due to the $e^{\pm i \omega t}$). The wavevector $|\psi \rangle_S$ evolves in time under the usual Schrodinger equation.
My goal is to write down the equations of motion for the "rotating frame" version of these operators. Namely I would just like to get the time evolution for $\sigma^-$ in the rotating frame. In fact, I'm quite sure the answer I should get is:
\begin{align}
\frac{d}{dt}\sigma^- = +i\Delta \sigma^- + i \frac{\Omega}{2}\sigma_z
\end{align}
I will present two approaches to try to derive this answer. One of them gets me to the correct answer. The other one does not. I want to understand why the second method doesn't get me the correct answer.
Unitary Transformations/Changing Frames
First a note on performing unitary transformations. I'll use the language of performing a unitary transformation with respect to a transformation operator $U_T$. I take the convetion that
\begin{align}
|\psi(t)\rangle_S = U |\psi(0)\rangle_S
\end{align}
Where $U$ is the time evolution operator in the Schrodinger picture. One can in general perform a unitary transformation on Hilbert space with respect to unitary operator $U_T$ by doing
\begin{align}
|\psi \rangle_S &\rightarrow U_T |\psi \rangle_S = |\psi\rangle_F\\
O_S &\rightarrow U_T O_S U_T^{\dagger} = O_F
\end{align}
$F$ indicates symbols in the transformed space. I'll write down the inverse transformations.
\begin{align}
|\psi \rangle_F &= U_T^{\dagger} |\psi \rangle_S\\
O_F &= U_T^{\dagger} O_S U_T
\end{align}
I'm going to try to make a general statement about transforming between different frames here. Suppose I am in one frame $A$ and I want to transform to another frame $B$ using transformation operator $U_T$. Note that I am preparing for a transformation FROM the Heisenberg picture into an interaction picture. Typically we change FROM the Schrodinger picture into another one so that is why I take time here to explore these transformations.
Suppose that in frame $A$ we have the following rules for time evolution.
\begin{align}
\frac{d}{dt}|\psi\rangle_A &= -\frac{i}{\hbar} H_{1,A} |\psi \rangle_A\\
\frac{d}{dt} O_A &= -\frac{i}{\hbar}[O_A,H_{2,A}] + \left(\frac{d}{dt}O_S \right)_A
\end{align}
The last term in the $O_A$ equation indicates that we should be able to rotate into the Schrodinger picture to check if $O_A$ has any "explicit" time dependence which needs to be accounted for in the time evolution. It's possible that my issue stems from a misunderstanding of this term.
Suppose that
$$
\frac{d}{dt} U_T = -\frac{i}{\hbar} T U_T
$$
I claim that if we transform into a new frame $B$ under the operator $U_T$ then we will get the new time dependence
\begin{align}
\frac{d}{dt}|\psi\rangle_B &= -\frac{i}{\hbar} (H_{1,B}-T_B) |\psi \rangle_B\\
\frac{d}{dt} O_B &= -\frac{i}{\hbar}[O_B,H_{2,B}+T_B] + \left(\left(\frac{d}{dt}O_S \right)_A\right)_B
\end{align}
In other words, you transform all of the operators but kets now evolve under $H_1-T$ whereas operators evolve under $H_2+T$. It is easy to see that the usual transformations FROM Schrodinger picture TO Heisenberg or Interaction picture are specializations of the transformation shown.
Heisenberg Picture
Finally I can show my two approaches to solve the problem. For both of them I first transform into the Heisenberg picture because, as I mentioned, I want an operator-centric solution. In this case we transform under the operator satisfying
$$
\frac{d}{dt} U_{SH} = -\frac{i}{\hbar} H_S U_{SH}
$$
We will find that kets have no time evolution while operators evolve under
\begin{align}
H_H &= \omega_0 \left(|e\rangle \langle e|\right)_H + \frac{\Omega}{2}\left(e^{+i\omega t} \sigma^-_H + e^{-i\omega t}\sigma^+_H \right)
\end{align}
First Approach
My first approach to solving this problem is as follows. First find the Heisenberg equation of motion for $\sigma^-_H$. This involves the commutator of $\sigma^-_H$ with $\sigma_{z,H}$ and $\sigma^+_H$. The result I get is
$$
\frac{d}{dt} \sigma^-_H = -i\omega_0 \sigma^-_H + i \frac{\Omega}{2} \sigma_{z,H} e^{-i\omega t}
$$
This is good except for the remaining $e^{-i\omega t}$ time dependence in the differential equation. The first approach to solving this problem is to forget any details about unitary transformations or "frames" and simply define the following operator
$$
\sigma^-_N = \sigma^-_H e^{+i\omega t}
$$
We might do this because we solved the free Hamiltonian problem in the Heisenberg picture earlier or because we know the answer to the problem.
We can then multiply both sides of the Heisenberg equation of motion by $e^{+i\omega t}$ and add a term to both sides to get
\begin{align}
e^{+i\omega t}\left(\frac{d}{dt}\sigma^-_H + i\omega \sigma^-_H \right) &= +i\Delta e^{+i\omega t}\sigma_-^H + i \frac{\Omega}{2} \sigma_{z,H}\\
\frac{d}{dt}\sigma^-_N &= +i\Delta \sigma^-_N + i\frac{\Omega}{2} \sigma_{z,N}
\end{align}
In the last line I have simply stated that $\sigma_{z,H} = \sigma_{z,N}$. I can justify this as follows (and this will segue into my second approach). We could have defined $\sigma^-_N$ as the result of the following unitary transformation.
\begin{align}
U_Z &= e^{+i \omega t (|e \rangle \langle e|)_H}\\
\sigma^-_N &= U_Z^{\dagger} \sigma^-_H U_Z = \sigma^-_H U_Z = \sigma^-_H e^{+i\omega t}\\
\sigma_{z,N} &= U_Z^{\dagger} \sigma_{z,H} U_Z = \sigma_{z,H}
\end{align}
So under this approach I am able to get the equation of motion I am looking for. I believe this approach is in fact correct.
Second Approach
The second approach is to apply the unitary transformation $U_Z$ above and directly derive the equations of motion for the operator in the resultant frame. I can pretty quickly explain the results I get. I'll denote the frame that arises here by $F$. When we perform the unitary transformation $U_Z$ we will find that $\sigma^-_F$ obeys
\begin{align}
\frac{d}{dt}\sigma^-_F = -\frac{i}{\hbar} \left[ \sigma^-_F, H_F -\omega (|e\rangle\langle e|)_F\right]
\end{align}
Again I've set the "explicit" time derivative part to zero.
My calculation is that 
\begin{align}
H_F -\omega (|e\rangle\langle e|)_F = - \Delta \left(|e\rangle \langle e|\right)_F + \frac{\Omega}{2}\left(e^{+i\omega t} \sigma^-_F + e^{-i\omega t}\sigma^+_F \right)
\end{align}
Notice that the effective Hamiltonian for the operator evolution STILL has factors of $e^{\pm i \omega t}$. This is the problem. This means the operator time evolution will still have these factors and that we haven't successfully factored out the time evolution as desired. When the commutators are calculated as above we get
\begin{align}
\frac{d}{dt}\sigma^-_F = +i\Delta \sigma^-_F + i\frac{\Omega}{2}\sigma_{z,F}e^{-i\omega t}
\end{align}
almost exactly as in the Heisenberg picture. It seems nothing has really been gained.
Question
Clearly something is discrepant between the two approaches. Namely, I believe that I performed the same unitary transformation in the two approaches so my suspicion is that $\sigma^-_N = \sigma^-_F$, however the equations of motion for $\sigma^-_N$ and $\sigma^-_F$ turned out different. The question is where have I made a mistake.
I'm open to the possibilities that I've done the rotating wave approximation wrong, that I've made sign errors somewhere, that I'm not treating explicit time dependence properly etc. I've been staring at this for a little while and I'm stumped.
Presumably the answer to where I've made a mistake will reveal what I need to do to finish the calculation I am attempting, but if not then I would like advice on how I can proceed.
Update
I think the solution may be related to the definition of $\Omega$ in terms of a dot product between the electric field and the dipole moment operator. In particular $\Omega$ is defined as something like
$$
\Omega = \langle g |\epsilon \cdot d|e\rangle
$$
I know that $|e\rangle$ evolves with a factor of $e^{\pm i \omega t}$ so perhaps this could come in to cancel the factor I am hoping to get rid of. I may be able to provide more details later but not at the moment.
 A: I think it would be interesting for you to know that this two-level system with this particular time dependence is exactly solvable and no rotating wave approximation is needed. 
Starting with the original Hamiltonian in Schrodinger representation $H$ and defining $H'=\lambda |e\rangle\langle e|$ -$\lambda$ is an undetermined constant- you can show (fill in the steps here)
$i\frac{d}{dt}|\Psi\rangle_I=(H-H')_I|\Psi\rangle_I$, where $A_I\equiv e^{iH't}A e^{-iH't}$ and $|\Psi\rangle_I\equiv e^{iH't}|\Psi\rangle$.
We made an interaction picture style transformation (but not exactly interaction picture!) to the vectors and operators  here in view of making the Hamiltonian into a time independent one. Here you can easily calculate the "interaction picture" Hamiltonian using the Baker-Hausdorff lemma:
$H_I= e^{iH't}H e^{-iH't}=\omega_0 |e\rangle\langle e|+\frac{\Omega}{2}(e^{i(\lambda-\omega)t}|e\rangle\langle g|+e^{-i(\lambda-\omega)t}|g\rangle\langle e|)$
and thus we find that:
$(H-H')_I=H_I-H'=(\omega_0-\lambda) |e\rangle\langle e|+\frac{\Omega}{2}(e^{i(\lambda-\omega)t}|e\rangle\langle g|+e^{-i(\lambda-\omega)t}|g\rangle\langle e|)$
If in the above equation one makes the smart choice $\lambda=\omega$, then all time dependence in the Hamiltonian is eliminated and and the Hamiltonian becomes the requested one.
$(H-H')_I=-\Delta |e\rangle\langle e|+\frac{\Omega}{2}(|e\rangle\langle g|+|g\rangle\langle e|)$
Note: As you can see this Hamiltonian transformation involved transforming the kets as well, so if you evolve the new Hamiltonian, you need to transform back to find the evolution of the original kets.
A: I have come up with an answer to this question. I've actually been toying with this problem for a very long time now and it ended up being much more of a bear than I ever imagined it might be. I had to finally learn some details of the Baker-Campbell-Hausdorff formula which I'd been able to avoid up until now.
If I have time I'll clean up the question, but I'll clarify here that my goal was always to derive the effective Hamiltonian which governs the time evolution of $\sigma^-_F$, not just the equations of motion for $\sigma^-_F$. I have already shown a way to derive the desired equations of motion in my first approach. Here I will show how to derive the desired ``rotating frame'' Hamiltonian. I'm curious to hear if anyone can answer this in a simpler way.
As I mentioned in my comment above the problem with the argument presented in the question is that in the second apporoach it is assumed that
$$
\frac{d}{dt} U_z = i\omega (|e \rangle \langle e|)_H U_z
$$
However, this would only be the case if $(|e \rangle \langle e|)_H$ were time independent. However, this is not the case because it is a Heisenberg operator that evolves under the given Hamiltonian. This led to the contradiction presented in the question above. The treatment must be adjusted.
However, it is still true that $\sigma_F$ as defined above is equal to $\sigma_N$ and thus $\sigma_F$ DOES have the correct equations of motion. This means that the $U_z$ defined above is the appropriate operator to transform us into the rotating frame, it will just be more difficult to tease out the Hamiltonian/equations of motion than I initially attempted.
It can be shown that when a unitary transformation of the sort given above is applied we can get the following equation of motion for the transformed operator.
$$
\tag{1}
\frac{d}{dt} \sigma^-_F = -i \left[\sigma^-_F, H_F + i U^{\dagger}_z \frac{d}{dt}U_z\right]
$$
This can be shown using the definition of $\sigma_F^-$ and the product rule for derivatives. The proof is for the reader, but similar proofs are found in the literature. The goal is to show that
$$
H_F + i U_z^{\dagger} \frac{d}{dt}U_z
$$
gives the desired Hamiltonian. Namely
$$
H_D = -\Delta (|e\rangle \langle e|)_F + \frac{\Omega}{2}\left(\sigma^-_F + \sigma^+_F \right)
$$
For brevity let me introduce the operator $\epsilon = |e\rangle \langle e|$. 
$H_F$ is easy to calculate. $H_F = U_z^{\dagger} H_H U_z$ so it is the same as $H_H$ but with all of the $H$'s replaced by $F$'s. The problem is that $H_F$ still contains explicit exponential time dependence. We will see that this is canceled by the second term.
We now set to work to calculate $\frac{d}{dt}U_z$. For reference I write down
$$
U_z = e^{i \omega \epsilon_H t}
$$
Recalling that $\epsilon_H$ is a time dependent Heisenberg operator. There is a Baker-Hausdorff formula which tells us how to calculate derivatives of exponential operators. I first came across it in this answer (note I think there is a mistake in the formula given here) and on this Wikipeida page. The formula is that
$$
\frac{d}{dt} e^{M(t)} = \sum_{n=0}^{\infty} \frac{1}{(n+1)!} \left[M(t),\left[\ldots\left[M(t),\frac{d}{dt}M(t)\right]\ldots \right]\right] e^{M(t)}
$$
Here there are $n$ commutators in each term. Specifically, there are enough commutation operations here such that the operator $M$ appears $n$ times.
In my case $M(t) = i \omega \epsilon_H t$. We need $\frac{d}{dt}M(t)$.
$$
\tag{2}
\frac{d}{dt}M(t) = i\omega\left(\epsilon_H + \frac{d}{dt}\epsilon_H t\right)
$$
I'll note that $M(t)$ will commute with the first term here so we can ignore it for all terms with $n>0$, however it will show up in the $n=0$ term. I'll ignore this part and add it back in at the end. We need to calculate the second term. Here is the first point where we need to plug in the Heisenberg Hamiltonian of interest for this specific problem to determine $\frac{d}{dt}\epsilon_H$.
$$
\frac{d}{dt}\epsilon_H = -i[\epsilon_H, H_H] = -i\frac{\Omega}{2}\left(e^{+i\omega t}(-\sigma^-_H) + e^{-i\omega t}\sigma^+_H \right)
$$
Now we are making progress. Note that since we have the property that $[\epsilon_H,\sigma_H^{\pm}] = \mp \sigma_H^{\pm}$ we will be able to calculate the chain of commutators above and end up with things proportional to $\sigma^{\pm}$
$$
\frac{d}{dt} U_z = \sum_{n=0}^{\infty} \frac{(i\omega t)^{n+1}}{(n+1)!} \left(-i\frac{\Omega}{2}\right)\left[\epsilon_H,\left[\ldots\left[\epsilon_H,e^{+i\omega t}(-\sigma^-_H) + e^{-i\omega t}\sigma^+_H\right]\ldots \right]\right] U_z
$$
We can split this into two separate commutator chains and apply the commutation $n$ times to find
$$
= \left(-i\frac{\Omega}{2}\right)\left[\sum_{n=0}^{\infty} \frac{(-i \omega t)^{n+1}}{(n+1)!} \sigma^-_H e^{+i\omega t} + \sum_{n=0}^{\infty} \frac{(i \omega t)^{n+1}}{(n+1)!} \sigma^+_H e^{-i\omega t}\right]U_z
$$
However we recognize these power series as $e^{\mp i\omega t}-1$. We can look ahead and see that the part coming from the $-1$ will cancel the pesky time dependence in $H_F$ that has bothered me for a very long time and the part coming from the $e^{\mp i \omega t}$ will cancel the time dependence appearing here resulting in the desired Hamiltonian. That is the answer there but for my own reference I'll carefully flesh out the details here. We also must recall to add back in the relevant part of $\frac{d}{dt}M(t)$ from $(2)$ which was ignored for the past few steps
$$
\frac{d}{dt}U_z = \left(-i\frac{\Omega}{2} \right)\left( \left(e^{-i\omega t}-1\right) \sigma^-_H e^{+i\omega t} + \left(e^{+i\omega t}-1\right)\sigma^+_H e^{-i\omega t}\right)U_z + i\omega \epsilon_HU_z
$$
$$
= -\left(-i\frac{\Omega}{2}\right)\left(\sigma^-_H e^{+i\omega t} + \sigma^+_H e^{-i\omega t} \right)U_z + \left(-i\frac{\Omega}{2}\right)\left(\sigma^-_H + \sigma^+_H \right)U_z + i\omega \epsilon_H U_z
$$
In the interest of building up $(1)$ I'll write out
$$
i U_z^{\dagger}\frac{d}{dt}U_z = -\frac{\Omega}{2}\left(\sigma^-_F e^{+i\omega t} + \sigma^+_F e^{-i\omega t} \right) + \frac{\Omega}{2}\left(\sigma^-_F  + \sigma^+_F \right) - \omega \epsilon_F
$$
The operators changed from $H$ to $F$ because they were conjugated by $U_z$.
Adding this to
$$
H_F = \omega_0 \epsilon_F + \frac{\Omega}{2}\left(\sigma^-_F e^{+i\omega t} + \sigma^+_F e^{-i\omega t} \right)
$$
We can finally see that
$$
\frac{d}{dt}\sigma^-_F = -i\left[\sigma^-_F,-\Delta \epsilon_F + \frac{\Omega}{2}\left(\sigma^-_F + \sigma^+_F \right) \right]
$$
$$
\frac{d}{dt}\sigma^-_F = -i\left[\sigma^-_F, H_D\right]
$$
So we see that $\sigma^-_F$ (and in fact any operator in this frame) evolves under the desired Hamiltonian. It is evident that this will lead to the desired equations of motion which are first order with constant coefficients and thus easier to work with than the time-dependent equations of motion.
