A system's density matrix can be written in the form you cite if and only if it is a classical mixture of factorisable pure states. A factorisable pure state is, of course, one that can be written as a tensor product $\psi_A\otimes\psi_B$, where $\psi_A$ and $\psi_B$ are pure states in subsystems $A$ and $B$, respectively. Correlations between measurements on subsystems $A$ and $B$ are indistinguishable from any other classical correlation between classical random variables, and, in particular, heed the Bell and CHSH inequalities. The classical probability theory of correlated random variables describes the joint probability densities for measurements on the two subsystems. You can think of these states as classical mixtures of block-diagonal stripes in the full quantum state space.
General entangled states, on the other hand, are those which are classical mixtures of more general, non-factorisable states. Correlations between measurements on the subsystems violate the Bell and CSCH inequalities.
Classical correlated states have a very special structure. Entangled states are the overwhelmingly likelier case, if you just chose a density matrix for the combined systems at random.