Yes, the phosphorous atom would become positively charged when it becomes ionized due to the ambient temperature and its electron wanders away, but that's not what is meant by a "hole". A "hole" is not simply a positively charged object in the lattice. You have to move beyond a classical picture and think in terms of quantum mechanics and electronic band theory, which is where the concept of "holes" really arises. "Holes" are empty states in nearly filled electronic bands. Due to their behavior under electric fields, these empty states can equivalently be thought of as mobile, positively charged quasi-particles (i.e., "holes") with characteristics dictated by the electronic bands. Your phosphorous ion is something completely different. True, it is positively charged, but it does not move according to the material's electronic band structure and in fact is not mobile at all. It's simply a static, immobile entity in the crystalline lattice.
Bottom line: "Holes" are mobile, positively charged quasi-particles whose behavior is determined by the electronic band structure of the material.