If a quantum state is pure why are its observables still probabilistic? As I understand it, a pure quantum state is one that can be represented as a ket $\lvert\psi\rangle$ in a Hilbert space, and it contains all the information about the state of the system. As such, we have complete information about the state of the system. 
Why then is the information we extract from the state $\lvert\psi\rangle$ still probabilistic? That is, why are the observables that we measure still associated with a probability? Is it simply that although we have complete information about the state of the system, by "complete information" it is meant that we know all the possible values that each given observable of the state can take?
One reason I ask, is that in the case of an open quantum system, we have to use mixed states in which we have incomplete information about the state of the system. This involves a classical probability since it is the lack of information that causes the probabilistic description of the state of the system, however there is still a "quantum probability" associated with such a system, like in the case of a pure state, since its observables have an intrinsic probabilistic nature. 
I find it confusing having the two, how to distinguish between them and why this so-called "quantum probability" arises, particularly when in the case of a closed or isolated quantum system we can use pure states which supposedly contain complete information about the observables of the system?!
 A: Your question has no answer. There is no "why".
It is a postulate of quantum mechanics that a pure state vector contains all information you can possibly know about a physical state. Note that this is not the same as saying it contains all information you can possibly imagine having about a physical state, particularly since your (and my) imagination is mostly classical - we humans are pretty bad at imagining states with no definite value for an observables.
The idea that this postulate of quantum mechanics might be wrong, that "complete information" means you can assign definite values to all observables and thus get a deterministic outcome for a single measurement, is the idea of hidden variable theories. Local hidden variable theories equivalent to quantum mechanics are excluded by Bell's theorem, non-local ones are possible, but might be even more philosophically dissatisfying.
Now, you also mention mixed states. Mixed states represent lack of knowledge about the quantum state of a system, they mean we are lacking information we could in prinicple have. In priniciple: Measure a system in a way that you obtain the values for a complete set of commuting observables, and you know the pure quantum state it is in. This is different from the probabilities for measurement of a pure state, because standard quantum mechanics says there is nothing even in principle you could know that would make you able to get rid of the probabilities.
However, there is an interesting fact: Quantum mechanics inherently comes with a process to generate the lack of information about subsystems - entanglement. If you have a pure state of a system that has subsystems that is entangled, the states you assign to the subsystems must be mixed states - it is the defining characteristic of entanglement that there are no single pure states you could assign to the subsystems. So this is another very un-classical feature: Having complete knowledge of the state of a system does not imply you have complete knowledge of the states of the subsystems. 
As I said at the beginning, there is no "why" we would currently know. but this should not really unsettle you - there are axioms in every physical theory that are taken for granted simply because they produce the correct observations. There is no answer to "Why do Newton's laws hold?" in Newtonian mechanics, or to "Why do electric and magnetic fields obey Maxwell's equations?" in electromagnetism, either.
A: Technically one cannot say observables are probabilistic, since they are mathematically described by deterministic operators. Now when an observable has different eigenvalues, then the Born rule is used to predict which value the experiment will get, and this is where probabilities arise. The Born rule is a postulate of Quantum Mechanics, historically proposed as an interpretation of the solutions to the Schrödinger equation, in order to fit experiments. See this discussion on the nature of the Born rule. 
Nobody knows what the fact that we need the Born rule means ontologically, although it seems clear that it cannot generally be said that an observable has any definite value when it is not being measured, which is confusing indeed.
