It's just a matter of doing the algebra right. We can take out a factor $\gamma$ from both factors in the expression:
$${V_{x}}'=\frac{d}{dt}\left(\gamma(x-\beta ct)\right)\left(\gamma + \frac{\beta \gamma}{c}{V_x}'\right)$$
because it's constant under differentiation w.r.t. $t$. This yields:
$${V_{x}}'=\frac{1}{1-\beta^2}\frac{d}{dt}\left(x-\beta ct\right)\left(1 + \frac{\beta }{c}{V_x}'\right)$$
The derivative can be evaluated:
$${V_{x}}'=\frac{1}{1-\beta^2}\left(V_x-\beta c\right)\left(1 + \frac{\beta }{c}{V_x}'\right)$$
All we need to do is solve this equation for ${V_{x}}'$. When working with large equations you need to be careful to prevent errors. The best way is to concentrate on the relevant parts of the equation, instead of trying to do everything at once. So, if we want to collect all the ${V_{x}}'$ terms then just concentrate on doing just that. On the left hand side ${V_{x}}'$ is present with a coefficient of $1$, on the right hand side it has a coefficient of:
$$A = \frac{1}{1-\beta^2}\left(V_x-\beta c\right)\frac{\beta }{c}$$
So, if we bring all the ${V_{x}}'$ terms to the left, it will get a coefficient of $1 - A$. The remaining term on the right hand side is:
$$B = \frac{1}{1-\beta^2}\left(V_x-\beta c\right)$$
So, let's see if we can simplify $1 - A$:
$$1-A = \frac{1}{1-\beta^2}\left(1-\beta^2 -V_x\frac{\beta}{c}+\beta^2\right) = \frac{1}{1-\beta^2}\left(1-V_x\frac{\beta}{c}\right)$$
Dividing both sides by $1-A$ yields:
$${V_{x}}'= \frac{V_x-\beta c}{1-\frac{\beta}{c} V_x}$$