Angular velocity from orientational displacement A 3-dimensional object is rotating around an unknown 3-dimensional axis through the object's center of mass. Its orientation is described by two unit vectors. I know an initial orientation and a final orientation and the time in between $\tau$. Four vectors in total. Let's call the initial orientation vectors $\vec{a_0}$ and $\vec{b_0}$, and the final vectors $\vec{a}$ and $\vec{b}$. Both vectors point out of the objects center of mass. How do I find the rotational velocity vector $\vec{\omega}$, where the direction is the rotation axis, and the length is the angular velocity in rad/sec according to the right hand rule?
 A: The rotation velocity vector $\vec \omega$ must be in the plane perpendicular to the vector difference $\vec a - \vec a_0$ and also be in the plane perpendicular to the vector difference $\vec b - \vec b_0$. Consequently, unless these two vector differences are parallel to each other (and provided neither is zero), the rotation velocity vector must be parallel to the cross product vector $ ((\vec a - \vec a_0) \times (\vec b - \vec b_0)) $; that is explicitly:  
$$ \vec \omega := \frac{ \| \vec \omega \| }{ \| ((\vec a - \vec a_0) \times (\vec b - \vec b_0)) \| } ((\vec a - \vec a_0) \times (\vec b - \vec b_0)).$$
Further, the "rotation speed" $\| \vec \omega \|$ can be determined through
$$ \frac{1}{\| \vec \omega \|} := \frac{\tau}{2~\pi}~\text{ArcCos}[~
\frac{ (((\| \vec \omega \|)^2~\vec a - \vec \omega (\vec a \cdot \vec \omega)) \cdot ((\| \vec \omega \|)^2~\vec a_0 - \vec \omega (\vec a_0 \cdot \vec \omega))) }{ (\| (\| \vec \omega \|)^2~\vec a - \vec \omega (\vec a \cdot \vec \omega) \|) (\| (\| \vec \omega \|)^2~\vec a_0 - \vec \omega (\vec a_0 \cdot \vec \omega) \|) }~],$$
or equally by a corresponding expression in terms of vectors $\vec b$ and $\vec b_0$.
Note that the argument of the $\text{ArcCos}$ function is a fraction which is independent of the magnitude of vector $\vec \omega$ (but depends on the direction of vector $\vec \omega$, which must be non-zero, of course). Therefore the above expression for the "rotation speed" $\| \vec \omega \|$ can be explicitly evaluated by substituting the cross product vector $ ((\vec a - \vec a_0) \times (\vec b - \vec b_0)) $ for $\vec \omega$ in the argument of the $\text{ArcCos}$ function, on the right-hand side of the expression.
This may be somewhat further simplified by noting that "rotation" means that
$(\vec a \cdot \vec a) = (\vec a_0 \cdot \vec a_0)$ as well as $(\vec a \cdot \vec \omega) = (\vec a_0 \cdot \vec \omega)$,
and likewise
$(\vec b \cdot \vec b) = (\vec b_0 \cdot \vec b_0)$ as well as $(\vec b \cdot \vec \omega) = (\vec b_0 \cdot \vec \omega)$;
but the resulting explicit expressions for the "rotation speed" $\| \vec \omega \|$ or even for the rotation velocity vector $\vec \omega$ still appear too complicated to be worth writing down here explicitly.
A: When a single vector is rotated from one angle to another, and the angle between them is not 180°, then there are infinitely many axes of rotation that could have brought that about - and these axes all lie on a plane that bisects the two vectors.
When you have two pairs of vectors, then for each pair there would be such a plane, and the actual axis of rotation will be the intersection of these planes.
It may not be possible to solve for both the axis and the magnitude of the rotation - in principle the problem is overconstrained. 
But let's assume that the problem is solvable (you have a real pair of vectors, and an unknown axis of rotation). We can then solve as follows.
We find the plane bisecting the two $a$ vectors, and the plane bisecting the $b$ vectors. The intersection of these planes is the axis of rotation. We should be able to find the magnitude of rotation by either considering $a$ or $b$ - and get the same answer. If we don't, we might use an "intermediate" value that gets us close to both.
The direction of the line of intersection $\vec z$ is normal to the normal of both planes. This requires taking lots of cross products. First we define
$$\vec{a_n} = \vec{a_0}\times\vec{a_1}$$
and
$$\vec{b_n} = \vec{b_0}\times\vec{b_1}$$
where the input vectors have been normalized (unit length). The cross product gives us a vector that is perpendicular to the pair of vectors: this is one of the vectors in each of the planes we want to intersect.
The second vector we need is the sum of the $a$ and $b$ vectors (which bisects the two):
$$\vec{a_s} =  \vec{a_0}+\vec{a_1}\\
\vec{b_s} =  \vec{b_0}+\vec{b_1}$$
Now we need the normals to the planes:
$$\vec{a_p}=\vec{a_s}\times\vec{a_n}\\
\vec{b_p}=\vec{b_s}\times\vec{b_n}$$
And finally the line that is the intersection of these planes is perpendicular to both these normals, so
$$\vec{z}=\vec{a_p}\times\vec{b_p}$$
This gets us the direction, but not the magnitude. I am not sure you can guarantee there is a unique magnitude - as I said above, the problem is overconstrained. But in principle you should be able to project the rotation needed (for example $\vec{a_n}$) onto $\vec{z}$ to get the correct scaling. But I think that you won't, in general, get the same value depending on whether you project onto $a$ or $b$.
A: Thank you both for the help. I were able to find a, better I think, solution myself, but only partially why it works. Maybe you could help me with that? If I break the rotation down into three components that each will satisfy the rotation for each vector, I can add the components together to find the total rotation axis.I have tested this in geogebra and it works.
The formula
$\vec{a}$ and $\vec{b}$ are both unit vectors and orthogonal to each other.
$$\mbox{I}:$$
$$\vec{c} = \vec{a} \times \vec{b}$$
$$\vec{c}_0 = \vec{a}_0 \times \vec{b}_0$$
$$\vec{a}_r = \vec{a}_0 \times \vec{a}$$
$$\vec{b}_r = \vec{b}_0 \times \vec{b}$$
$$\vec{c}_r = \vec{c}_0 \times \vec{c}$$
$$\vec{\omega}_{dir} = \vec{a}_r + \vec{b}_r + \vec{c}_r$$
$$\mbox{II}:$$
$$|\vec{\omega}_{dir}| = 2\sin{\alpha}$$
$$\vec{\omega} = \frac{1}{\tau}\frac{\vec{\omega}_{dir}}{|\vec{\omega}_{dir}|} \arcsin(\frac{|\vec{\omega}_{dir}|}{2})$$
$\vec{\omega}$ is the axis through the object center of mass, that it rotates around according to a right hand coordinate system. $|\vec{\omega}| = \alpha$ and $\alpha$ is the angle the object rotates around the axis. This formula works only when $\alpha \leq 90^\circ$.
How it works
Or rather how I came up with the formula. I don't quite understand how it works myself.
$$\mbox{I}:$$
Firstly I define a vector $\vec{c}$, which is also a unit vector and orthogonal to $\vec{a}$ and $\vec{b}$. $\vec{a}$, $\vec{b}$ and $\vec{c}$ can then be defined as the local coordinate system of the object. Let $\vec{x}$, $\vec{y}$ and $\vec{z}$ be the unit vectors describing the world coordinate-system, all three orthogonal. I started of with imagining the situation where $\vec{a}$ and $\vec{b}$ where in the $\vec{x}$$\vec{y}$-plane and the rotation axis ($\vec{\omega}$) pointing in the $\vec{z}$ direction. The cross product between the initial and final positions of $\vec{a}$ ($\vec{a}_r$), and the same for $\vec{b}$ will now point in the right direction (the direction of $\vec{\omega}$), and so will their sum. $\vec{c}_r$ will be zero, so if I throw that in as an addent, for symmetry reasons, the answer will be the same for this situation. Let's call the sum $\vec{\omega}_{dir}$. If I reverse the rotation, $\vec{\omega}_{dir}$ will reverse its direction, so till now it's looking good.
If I rotate the situation, it will also work. For instance if $\vec{b}$ and $\vec{c}$ were in, for instance, the $\vec{y}$$\vec{z}$-plane. Or any other combination.
I also tested situations in-between in geogebra, and compared it with Rodrigues' rotation formula. It gave the right result, but I don't know why.
$$\mbox{II}:$$
To figure the magnitude of the rotation ($|\vec{\omega}_{dir}|$), I thought of the; $\vec{a}$ and $\vec{b}$ in the $\vec{x}$$\vec{y}$-plane situation. Due to the nature of the cross product, we see that $|\vec{a}_r|$ and $|\vec{b}_r|$ must be $\sin{\alpha}$. Therefore the length of the sum must be $2\sin{\alpha}$. This also works in all situations according to my experiments.
