This picture is not wrong, but it involves a hidden simplification that makes it not quite right if taken literally.

You see, the random black ball / gray ball is what you get if you perform a measurement on the thing that comes out of the Hadamard gate, but that's not its direct output.
What comes out on the other side is a superposition state (a "gray" ball, if you will) - which is not some purely conceptual thing, it's a separate, third state (out of many more - there's a continuum of states; black, white and 50%/50% gray are just three out of many possibilities).
However, the "gray" state cannot be detected as such by the measuring device used; the measurement causes the state to "collapse" to white or black, in this case with 50% chance for either outcome. (With a measuring device constructed differently, you could detect this "gray" state, and a different kind of "gray" state, but not black or white - a measurement on, say, the black state would randomly collapse the qubit to one of the two distinct grays.)
The states of a qubit can perhaps be better understood as a clockface (with only one handle). In that picture, the qubit is in the black state when the handle is pointing to the right, to 3 o'clock, and in the white state when the handle is pointing straight up, to 12 o'clock. We say these states are orthogonal (literally, the state vectors, represented by the clock handle positions, are orthogonal)1.
1 There's a slight subtlety in that the clock handle can point in the exact opposite (negative) direction too; 9 o'clock is also measured as the black state, and 6 o'clock is also measured as the white state; there's no way to distinguish these on measurement, but the states are different, and they may affect how the qubit is transformed as it goes through different quantum logic gates.

Now, a qubit is in a superposition state, a "gray" state, when the clock handle points anywhere else - so, there's a whole bunch of superposition states, all of which correspond to different probabilities for the qubit to collapse to black or white after measurement (again, think of these as of different shades of gray). If the clock handle is closer to the horizontal axis, then the measurement is more likely to result in the black state, and if it is closer to the vertical axis, the white state is more likely. If the clock handle is at 45 deg. angle, the chances are 50/50.
(source)
What the Hadamard gate does (in this analogy2) is it moves the clock handle by 45 degrees in the clockwise direction, and then mirrors the result across the horizontal axis (flipping the orientation of the vector vertically). So, if the state was black to begin with, this operation makes it point to half past 1 (45 deg. angle); what comes out of the gate is depicted in the image above (the arrow with the $|\psi\rangle$ label). This is why the measurement outcomes are random with equal probability.
2 I say "in this analogy" because there's also a different way to depict a qubit, called the Bloch sphere, where this operation corresponds to a different angle. However, the clockface analogy is IMO better for developing the initial intuition.
Now, what actually happens when you have two Hadamard gates in a series is that this rotate & flip operation happens twice in a row, and it just so happens that doing this gets you back to where you started - it's just the nature of the operation itself. A qubit comes out of the first gate in a superposition state, no measurement is performed, so it stays like that ("gray") before entering the other gate, and after that it is transformed back to the original state.
So, there's no extra hidden variable in the qubit itself, and the Hadamard gates have no "memory cells" to remember what the qubit's state was initially. It just that the operation itself is such that double application is the same, in its result, as doing nothing. Much like how, if you spun around your own body axis by 180 deg twice, you'd get back to your original orientation.
In linear algebra terms, this logic gate corresponds to the order 2 Hadamard matrix, and composing those results in the identity matrix (which does nothing when acting on a state vector):
$$
H = \frac{1}{\sqrt 2} \begin{bmatrix}
1 & 1 \\
1 & -1 \\
\end{bmatrix}
$$
$$
HH = \frac{1}{\sqrt 2} \frac{1}{\sqrt 2} \begin{bmatrix}
1 & 1 \\
1 & -1 \\
\end{bmatrix}
\begin{bmatrix}
1 & 1 \\
1 & -1 \\
\end{bmatrix}
=
\begin{bmatrix}
1 & 0 \\
0 & 1 \\
\end{bmatrix}
$$