I'd say that your interpretation is basically right. Just keep in mind that a hole is simply an absence of an electron, so you can view any problem from the electron or hole point of view.
So, in the n-type example, one way of looking at it is that there are no holes in the VB because the VB is full of electrons. I.e. there is no absence of electrons, so there are no holes. In the CB, there are some electrons.
In the p-type example, one way of looking at it is that there are holes in the VB because there is an absence of electrons. You could either say that there are no electrons in the CB (which is what most people would say) or that the CB is full of holes (which is still a correct thing to say.)
EDIT: To clarify the point that we are free to make the choice about wheter to use electrons or holes, I quote Ashcroft and Mermin ch 12, p 226:
"Thus the current produced by occupying with electrons a specified set of levels is precisely the same as the current that would be produced if (a) the specified levels were unoccupied and (b) all other levels in the band were occupied but with particles of charge +e (opposite to the electronic charge.)
"Thus, even thought the only charge carriers are electrons, we may, whenever it is convenient, consider the current to be carried entirely by fictitious particles of positive charge that fill all those levels in the band that are unoccupied by electrons. The fictitious particles are called holes."
EDIT 2: With regards to charge neutrality. If you commit to the full electron/hole picture, then holes are the positive charges of the ions, so charge neutrality would mean an equal number of holes and electrons. It is easy to accidentally double count positive charges by naively counting ions and holes separately.
To be fair, you can count holes and ions carefully and adopt a hybrid electron/hole/ion point of view that holes only exist in the VB and the levels in the CB are not occupied by anything (holes or electrons), but in some ways that is more complicated since you now have to keep track of three particles (ions, holes, and electrons) rather than just two (electrons and holes).
In any case, the hybrid electron/hole/ion point of view is no more or less valid than the full electron/hole point of view, where all unoccupied electron levels are filled with holes.