In an idealized semiconductor at absolute zero, all the valence states are occupied by electrons, and all the conduction states are empty. When you take one electron an place it into the conduction band, you leave behind a state that is no longer occupied in the valence band.
Now, lets say you have a sample with a billion electons, all in the valence band, and you pop one into the conduction band. Figuring out how that one electron can move about in the conduction band is easy - there isn't anything else there to worry about. For the valence band, you could now worry about how the 999,999,999 electrons move, or you could instead say, wow, there is one empty state in a sea of full states - wouldn't it be easier to figure out how the empty state moves and focus on just that one?
This is a common technique in physics, be it called renormalization, or quasi-particles, or whatnot - reframe the problem to make it simpler.
So, the "hole" is what we call the empty state - there is an electron "missing" in the valence band, and we watch it move around. To avoid saying 'the state not occupied by an electron' lots of times, it got shortened to 'hole'.