Is the voltage ever undefined?
The voltage, or potential difference, between any two points is always defined. It is defined as the work per unit charge required to move the charge between the two points.
"The voltage between two points that are not part of the same circuit
is undefined".
It is not defined in the sense that one cannot determine what that voltage is by applying Kirchhoff's voltage laws to two independent circuits. But any voltage that may exist between them is still defined as above.
Or in other words, "if I have a circuit that is fully isolated
("floating"), then the voltage between any point in it and the ground,
is undefined".
I look at the question of voltage between a point isolated from ground and ground as a separate one. There is a known electric field between an isolated point in the atmosphere and earth ground. In areas of fair weather, the atmospheric electric field near the surface of the earth's surface typically is about 100 V/m directed vertically in such a sense as to drive positive charges downward to the earth. See:
https://glossary.ametsoc.org/wiki/Atmospheric_electric_field#:~:text=In%20areas%20of%20fair%20weather,charges%20downward%20to%20the%20earth
So if a point in the circuit is 1 meter above the ground, the voltage between it and the ground with air in between would be 100 V. However, due to air being a good insulator, the current density is only about 10 $^{-12}$ amperes per square meter parallel to the earth, so its influence on the circuit would be appear to be negligible. Its effect on you would also be negligible since you are a relative good conductor (relative to the air) so that when you stand on the ground you and the ground become equipotential surfaces. See.
https://www.feynmanlectures.caltech.edu/II_09.html
Regarding your follow up question if the two metal pieces have the same positive or negative charge, the voltage would be zero. If you connected the pieces with a conductor there would be no current in the conductor.
Hope this helps.