The intuition that a black hole must have a very large mass is not true. The relevant parameter is how much mass is there within a (volume of some characteristic) radius. In the case of simple spherical objects, if a mass $M$ is concentrated within a radius $2GM/c^2$ then light (or anything else for that matter) cannot escape from the region $r< 2GM/c^2$, and the region $r< 2GM/c^2$ is called a black hole. Thus, for any small amount of mass, if it is concentrated within a (volume characterized by a) small enough radius, then it is a black hole. In principle, you can have a black hole of the mass of a human being, but of course, its radius would be ridiculously small. This doesn't mean that all astrophysical stars, no matter their mass, would turn into black holes because the non-gravitational forces within the stars can resist the mass of the star from concentrating up to the required small radius $2GM/c^2$ if the mass of the star is not large enough. However, if the mass of the star is large enough (as described by the Chandrashekhar Tolman-Oppenheimer-Volkoff limit$^*$), the mass of the star would reach a stage where it's confined within the radius $2GM/c^2$, and it would become a black hole.
Edit
Notice that the relevant parameter is $M/r$, not $M/r^3$. The radius of a (non-rotating uncharged) black hole with mass $M$ scales as $r_s\sim M$. In other words, the density of a black hole with mass $M$ scales as $ M/r^3_s\sim 1/M^2$. Thus, if you have a black hole with small enough mass (which would correspond to a black hole with a small enough radius), you can get as high a density as you want. There is no fundamental restriction on the maximum density as such besides whatever restrictions might exist on how small you can make a black hole in a quantum theory of gravity.
$^*$ Thanks to @CharlesFrancis
for this correction. The Chandrashekhar limit is the limit on the maximum mass of a stable white dwarf which can either devolve into a neutron star or a black hole if the mass is higher than this limit. However, the Tolman-Oppenheimer-Volkoff limit is the limit on the maximum mass of a neutron star beyond which it would devolve into a black hole.