First of all you can directly write $p$ as a function of $V$ without solving a quadratic equation:
$$ p = \frac{N k_B T}{V- N b } - a \frac{N}{V} $$
Then you take a look at the interesting points
$$\left(\frac{\partial p}{\partial V}\right)_{T,N} = 0 $$
The solutions to this equation tell you where the bulk modulus (or its inverse, the compressibility) might change the sign. That's how you can use this criterion to determine the boundaries of unstable regions.
EDIT: If I am not mistaken, the equation above has solutions
$$ \vartheta_\pm = b \frac{1}{1\pm \sqrt{\frac{k_B T}{a}}} = b \frac{1}{1\pm \sqrt{t}}$$,
with $\vartheta = V/N$ and the reduced temperature $t=T k_B /a $. In terms of this reduced variable and $v = \vartheta / b$ we can write
$$ \left(\frac{\partial p}{\partial \vartheta}\right)_{T,N} = \frac{a}{b^2} \left(\frac{1}{v^2}- \frac{t}{(v-1)^2} \right) \overset{!}{\leq} 0$$
So you know the critical points where the sign might change and you know the equation for checking which sign is present. The only thing you have left do to do is check which sign is present in which region along the $\vartheta$ axis. This is only math left to do.. Since you know the sign along the the $\vartheta$ axis you will automatically know the sign along the $\vartheta^{-1}$ axis.