E.g. putting your finger close to the Generator and seeing a spark between it and your finger. Obviously this does not kill you, but why is there such low current produced when the potential difference is so great?
The statement "Obviously this does not kill you" is not true in all cases and certainly not true for larger Van de Graaff machines.
The charge on a VdG is stored on the dome whose capacitance is approximately given by $C = 4 \pi \epsilon_o R$ where $R$ is the radius of the dome which for a dome of radius 20 cm is approximately 20 pF.
The charge stored is $Q= CV$ where $Q$ is the charge on the dome and $V$ the potential difference between the dome and earth so for a potential difference of 100 kV this represents a charge of 2 $\mu$C. The charge transferred by a current of 2 $\mu$A in one second.
Increasing the voltage brings problems of leakage of charge down the supports which hold up the dome and the air around the dome where there will be corona discharge. You can often hear this above the noise of the motor driving the belt. Large VdG machines are usually housed in a vessel which contains gas under high pressure to reduce corona discharge or even spark loss. You will also have noted that high voltage devices tend not to have any sharp edges.
Then you have the belt having a limited capacity as to the amount of charge it can transfer from the bottom of the VdG machine to the top. It also will suffer leakage. You may have noticed that on a small VdG the motor slows down as the charge (and hence potential difference) builds up because the motor has to do more work lifting the charges against a potential gradient.
Current and voltage are different things. Any source of voltage be it a battery, solar cell, wall-wart power supply, your wall outlet, some overhead transmission lines or lightening has a limit on how much current it can supply.This is obvious when you think about it, if there was no limit you could do spot welding of heavy steel plates with a few AAA cells. Why is there a limit ? Because the number of electrons stored (or available) in the power source has a finite limit based upon how many have been "stuffed" in there AND the current is also limited by the resistance of the circuit. In the case of the Vann De Graaff we are stuffing electrons up into the top dome as we wind the handle. It is really a capacitor you are charging up and the more electrons on the top plate the higher the voltage. The capacitance is low though (maybe a few pF ?) so there is a low (relative, still millions) limit to how many electrons are up there before the voltage breaks down.
Wind the V de G up to the max, just before breakdown, and you have it fully charged and can make a nice spark, it will last for a tiny tiny fraction of a second and the current will be low because there are just not enough electrons flowing to make it higher because there are not enough electrons there in the first place.Current is a measure of flow rate of electrons.
If you could find, or make, a more substantial capacitor with a voltage rating higher than the V de G you could (if your arm doesn't tire) charge that too and get a higher current when it is discharged. From this we can see that the higher the voltage on a capacitor and the higher the capacitance the more dangerous it might be if you discharge it through a low resistance (or yourself). People have been killed & injured from capacitor discharges on so called "dead" i.e. disconnected/isolated electrical equipment.
W=1/2 CV^2 Try googling some more capacitor equations to understand the relationships between Capacitance, Voltage, Energy Stored, Discharge Times, Etc.
Ohm's Law sets the intensity of current (I) directly proportional to potential difference (V) and indirectly proportional to resistance (R). Current is expressed in amperes, potential difference in volts, and resistance in ohms:
I = V / R
Current will be higher as resistance is lower. While Van de Graaf generators can produce a large potential difference (high voltage), they do this in an environment of high resistance, which is necessary to allow charge to accumulate in the generator.
Smaller Van de Graaf generators with limited storage capacity generally discharge before accumulating a potential difference great enough to kill you. Just as static electricity builds up in your clothing as you shuffle across a carpet and discharges with an impressive spark when a sharp edge of your body comes near a ground, but doesn't seriously injure you, a small Van de Graaf generator doesn't have great enough storage capacity to do more than produce an impressive spark.
Also, as Farcher points out in his answer, power demand of the moving belt mechanism used in small Van de Graaf generators to deposit charge in the storage dome, becomes greater as charge accumulates, which limits the ability of the mechanism to deposit charge in the dome. Van de Graaf generators are dependent on high R to operate, and run into diminishing returns when building up high V. So it's difficult to raise V/R to achieve a high current relative to the power consumption of the mechanism.