The CMB was emitted at an energy of $E_{em}=13.6\text{ eV}$, which is the binding energy of hydrogen. This corresponds to a wavelength of $$ \lambda_{em} = \frac{hc}{E_{em}} \approx 9.12\times 10^{-8}\text{ m}$$
Redshift can be calculated by $$ 1+z = \frac{\lambda_{obs}}{\lambda_{em}} $$
If we observe blue light at 400 nm, we get a corresponding redshift of about $z_{blue} = 3.4$. For red light at 700 nm, we get $z_{red} = 6.7$.
The scale factor of the universe (the amount by which it has expanded) is related to redshift by $$ \frac{a_\text{now}}{a_\text{then}} = 1+z $$
For a flat, matter-dominated universe $$ \frac{a_\text{now}}{a_\text{then}} = \left ( \frac{3 H_0 t}{2} \right ) ^{\!2/3} $$ with $H_0$ the Hubble parameter today and $t$ the age of the universe. We solve to get $$ t = \frac{2}{3 H_0} \frac{1}{(1+z)^{3/2}} $$
We get $t(z_{red}) \approx 425$ million years, and $t(z_{blue}) \approx 980$ million years.
So according to this model the CMB was in the visible spectrum from about 13.2 - 12.8 bilion years ago.