Old-fashioned 'analogue' voltmeters (with a pointer and scale) did require a small current through them in order to work. They consisted of a sensitive ammeter (current meter) in series with a large resistance. They measured potential difference indirectly by using the fact that the current was proportional to the pd across the resistor (Ohms's law). Roughly speaking, the more you paid for a voltmeter, the more sensitive its ammeter, so the resistance could be higher, and the meter would have less effect on the pd you were trying to measure!
Digital voltmeters work on a totally different principle – and one that's much harder to explain! In theory they require no current to pass through them. First a very useful analogy: if we compare an electric current to the rate of flow of water through a pipe from a high reservoir to a lower reservoir, then we can compare potential difference to the height difference between the water levels in the reservoirs. Height difference is proportional to difference in gravitational PE per unit mass; Potential difference is proportional to difference in electrical potential energy per unit charge.
Modern voltmeters produce their own internal potential difference that rises at a constant rate (think about a water level rising) from zero. At the instant when the internal pd starts to rise, an internal clock is started. There is an electronic device that detects when the internally generated pd is equal to the unknown pd that you've applied to the voltmeter (water levels equal!). The device then stops the clock. The longer the clock has been going, the higher the unknown voltage must have been. The digital display of the voltage is made to be proportional to this elapsed time. The process repeats over and over again, keeping track of any (not too fast) changes in the unknown voltage.
Hope that you've been able to follow some of this!