In common introductory textbooks, voltage is defined as the electric potential difference (say, between A and B) between two points in space. They say that A and B have a potential difference, and both A and B are at unique electric potentials.
This is completely narrow viewpoint.
Electric potential, electric potential difference, and voltage are all completely synonymous.
In your case, you are right that it doesn't make sense to define a unique potential at point A. When you say the "potential at point A", what you really mean is the difference in electric potential between point A, and some explicit (e.g., circuital ground) or even implied (e.g., infinity) point of reference.
After all, no potential is absolute. When you say a ball "has potential energy $mgh$", you are secretly saying that "the gravitational potential energy difference between its height and the ground is $mgh$". You could also say that its gravitational potential is zero at that height, in which case it would have potential $-mgh$ at the ground.
It's all relative to one's choice of reference point -- and this choice is always made, either explicitly or implicitly.