A thorough understanding of holes, why they are useful and why classical analogies fail requires a rudimentary understanding of energy band structures in solids. The classical "bubble" picture alone, while it might be useful for introducing the idea of a hole, fails to explain the positive Hall or Seebeck coefficients of p-type semiconductors (and some metals).
Semiconductors (and some metals) have a valence band that is mostly filled, except at the very "top" (highest energy states) where some vacant electron states may exist. Electrons near the top of the valence band have the peculiar property that when a force acts upon them, they accelerate in the opposite direction: they have a negative effective mass.
Since the forces due to electric and magnetic fields are proportional to charge, valence band electrons thus respond to forces as if they were positively charged. Moreover, each vacant state (or "bubble") gets dragged along with nearby electrons, and moves precisely as if it were occupied by an electron, i.e. like a positive charge. The common simplified explanation (like the analogy in A. I. Breveleri's answer) gets this wrong when it claims the electrons move in one direction and the "bubbles" move in the opposite direction.
Suppose the current density due to the valence band electrons is $\vec J_\text{occupied}$. Let's define $\vec J_\text{vacant}$ as the current density that the vacant valence band states would yield if they were occupied. When the valence band is completely filled, for each electron moving in one direction, there is another moving in the opposite direction, so the net current is zero:
$$\vec J_\text{occupied}+\vec J_\text{vacant}=0$$
$$\vec J_\text{occupied} = -\vec J_\text{vacant}.$$
This says that the valence band current is the negative of the current that would result from electrons if they occupied the vacant states. We can thus regard this current as being due to positively charged particles occupying the states free of electrons. Since there are much fewer vacant states in the valence band, they are so much easier to keep track of in calculations.
In summary, vacant states in the valence band move as if they were positively charged particles, and contribute to current as if they were positively charged particles, and consequently it is convenient to attribute the dynamics and the current of the valence band to fictitious positively charged particles.