What happens to the position function when an oscillator is overdamped and does not have angular frequency? My question is simple: What happens to the behavior of the position function, $x(t)$, when an oscillator is overdamped and $\omega$ does not exist? 
Here's the background on why I'm confused:
For an overdamped spring, we have the two forces $F_{res}=-kx$ and $F_{damp}=-bv$. Since $\Sigma F=ma$, we have the second-order differential equation: $$-kx-bv=ma\\-kx-bx'=mx''$$ (where $x$ is the position of the spring). This models the behavior of a damped spring. From this, we derive the position function: $$\displaystyle x(t)=Ae^{\frac{-b}{2m}t}\cos(\omega t-\phi)$$ and formula for the value of $\omega$: $$\omega=\sqrt{\frac{k}{m}-\left(\frac{b}{2m}\right)^2}=\sqrt{\frac{4km-b^2}{4m^2}}$$
However, $\omega$ has a domain restriction. $4km\geq b^2$ - if not, then $\omega$ doesn't exist.
So, what happens to the behavior of $x(t)$ if $4km < b^2$?
The reason I'm perplexed by this is because clearly $\omega$ doesn't exist - this is fine, because when $4km < b^2$, the oscillator is overdamped. But, the position function $x(t)$ relies on the existence of $\omega$; if it does not exist, then $\cos(\omega t - \phi)$ doesn't exist, so the position function is invalid.
But the behavior dynamic still holds; the differential equation still makes sense, but the solution does not. 
How is this behavior supposed to be modeled?
 A: What you wrote is a solution, it is just not the only form of the solution.
Here are a couple ways to proceed:


*

*What happens when $4km < b^2$ ? Let's march ahead done this road.


The value in the square root becomes negative, and so there is no real solution for $\omega$.  However you can find imaginary solutions for $\omega$.
$$\omega = \frac{\pm i}{2m} \sqrt{b^2 - 4km} = \pm i K$$
where $K$ is a real value.
But to evaluate this in the equation, you need to know how to calculate the cosine with an imaginary argument. This can be done with Euler's formula,
$$ e^{i\theta} = \cos(\theta) + i \sin(\theta) $$
or using the symmetry of cosine to solve for that
$$ \cos(\theta) = \frac{1}{2}(e^{i\theta} + e^{-i\theta}) $$
Plugging in our imaginary $\omega=iK$:
$$ \cos(\omega t - \phi) = \frac{1}{2}(e^{i(\omega t - \phi)} + e^{-i(\omega t - \phi)}) \\
= \frac{1}{2}(e^{i(iK t - \phi)} + e^{-i(i K t - \phi)}) \\
= \frac{1}{2}(e^{- Kt}e^{- i \phi} + e^{Kt}e^{i\phi})
$$
So we get terms exponentially decaying or growing.  That is what happens when $\omega$ becomes imaginary, it goes from an oscillatory equation to a decay equation.
Notice your equation of motion is linear in $x$.  So we can take a linear combination of any two solutions and get another one.  This along with initial conditions and the requirement that the solution is real, will (after messy math) lead to the correct solution.


*Alternatively, we can decide that no real $\omega$ means we don't have a solution of the correct form, and we should look for other solutions.  We already have a hint from above.


$$ x = A \exp(-rt)$$
Plug in our guessed solution to the differential equation
$$mx'' + bx' + kx=0$$
$$mr^2x+b(-r)x + kx=0$$
which gives the constraint on $r$
$$mr^2-br + k=0$$
$$r = \frac{-(-b) \pm \sqrt{b^2 - 4mk}}{2m}$$
So now we have the exact opposite constraint $b^2-4mk\ge 0$. Our imaginary oscillatory rate is now a decay rate!
Since we can add solutions to get a new solution, the most general is:
$$ x = A \exp(-r_1t) + B \exp(-r_2t)$$
Where $r_1,r_2$ are the two solutions to the constraint on $r$ (given by the $\pm$ in the quadratic formula).
A: To add to CuriousKev's answer, another way of looking a linear differential equation is by looking at their eigenvalues. The order of the differential equation determines hoe many eigenvalues there are. So like you stated this is a second-order differential equation, therefore there will be two eigenvalues, which I will call $\lambda_1$ and $\lambda_2$. The equation of motion then becomes
$$
x(t) = C_1 e^{\lambda_1 t} + C_2 e^{\lambda_2 t},
$$
where $C_1$ and $C_2$ will be determined by your boundary conditions. This equation is not entirely the general equation of motion, since when $\lambda_1=\lambda_2=\lambda$, then $C_1$ and $C_2$ will not be able to satisfy all boundary conditions. In that case the equation of motion will become
$$
x(t) = \left(C_1 + C_2 t\right) e^{\lambda t}.
$$
The eigenvalues can be found by rewriting the differential equation as a matrix product, where the eigenvalues are equal to the eigenvalues of the matrix,
$$
\dot{\vec{x}} = A \vec{x},
$$
where in your case $\vec{x} = \begin{bmatrix}x & \dot{x}\end{bmatrix}^T$, thus $\dot{\vec{x}} = \begin{bmatrix}\dot{x} & \ddot{x}\end{bmatrix}^T$. From this the matrix $A$ can be derived to be
$$
A = \frac{1}{m}\begin{bmatrix}0 & m\\ -k & -b\end{bmatrix}.
$$
The resulting eigenvalues will therefore be
$$
\lambda_1 = \frac{-b}{2m} + \sqrt{\left(\frac{b}{2m}\right)^2-\frac{k}{m}} \bigvee \lambda_2 = \frac{-b}{2m} - \sqrt{\left(\frac{b}{2m}\right)^2-\frac{k}{m}}.
$$
Note that these eigenvalues can be complex, but if they are and $k$ and $b$ are real then they will be each others conjugate, such that imaginary parts of the equation of motion will cancel out. For higher order linear differential equations complex eigenvalues will always come in pairs.
