$du,dv$ are light-like, i.e. they could, in principle, be viewed as some affine parameters of some light-rays. However, we will focus (in the spirit of the usual coordinate-nature analysis) on what is the nature of either $u,v$ constant. I.e., we want to know what is the nature of $u,v$ constant hypersurfaces and derive the nomenclature from this.
We will take it as fact that u,v can be viewed as parametrizing some light congruence.
Now the question is what is the nature of congruence parametrized by $u,v$. Let us consider some finite $t=t_0$ and $r_*=r_0$. If we are investigating $u,v$ constant hypersurfaces, then $t>t_0$ means surely $r_*>r_0$ for $u=t-r_*$ constant. I.e., the $u=const.$ surface is farther out at a later time. Hence, the lightcone is interpreted as outgoing with respect to the centre $r_*=0$ for $u=$const. and we call the respective Finkelstein coordinate the outgoing coordinate.
The opposite is true for $v=t+r_*$. $t>t_0$ means surely $r_*<r_0$ for $v=$ const., or that for $v$ constant we have a lightcone ingoing with respect to the centre $r_*=0$.