Why do we use the whole radius vector in the angular momentum, and not just the unit vector? Why don't we only use the unit vector of the radius? 
I was thinking that, all we need to know is the component of the linear momentum, perpendicular to it, so why use the magnitude of the radius vector too?
 A: The answer to this is true for all definitions in physics. You are asking why angular momentum is defined as $\mathbf L=\mathbf r\times\mathbf p$ instead of $\mathbf L=\hat r\times\mathbf p$. The answer is because the former is very useful. In other words, not only is it useful to know the orientation of the momentum $\mathbf p$ relative to the origin, it is also important to know how far away from the origin you are.
I am sure there are many, many ways to show why the distance is important. One such way is through torque $\boldsymbol\tau=\mathbf r\times\mathbf F$. I am sure you can think of some real-world examples where using the entire $\mathbf r$ vector is important; for example, when opening a door, is it better to push on the door closer or farther from the hinge? Is it better to push perpendicular or parallel to the door? Torque and angular momentum are related through a time derivative: the rate of change of the total angular momentum of a system is equal to the external torque acting on it i.e. $\boldsymbol\tau=\mathbf{\dot L}$. This relation would not hold if the latter proposed angular momentum was used.
Would it be wrong to try and use what you propose? Certainly not. Would it be useful? I am not sure, but if it is you would need to show why it is useful. Either way, you seem to be going backwards. You seem to have an idea of how angular momentum "should be", but this isn't how we determine useful quantities in physics. It would be like me saying I want to define a new concept called "ultra-fine momentum", and I want to come up with an equation for it. This doesn't make sense, because how would I know what is the "right definition" for this random thing I am deciding to use? 
In other words, the question isn't, "I want to define angular momentum, what should the definition be?" The question is, "This thing here, $\mathbf r\times\mathbf p$, keeps popping up. Could this be an important/fundamental quantity for certain systems?" It turns out that keeping the entire $\mathbf r$ vector in this definition turns out to be very useful for many systems across many branches of physics, so we keep it like that and we have given it the name "angular momentum".
A: Angular momentum is the moment of momentum just as torque is the moment of force and velocity is the moment of rotation.
All the moment of xxx  quantities are evaluated using the full radius vector since the moment arm (minimum perpendicular distance) to the line where the quantity is acting though is needed.
$$\matrix{ \text{(moment of rotation)} = \boldsymbol{r} \times \text{(rotation)} & \} &
 \boldsymbol{v} = \boldsymbol{r} \times \boldsymbol{\omega} & \} & v = d_\perp\, \omega }$$
$$\matrix{ \text{(moment of momentum)} = \boldsymbol{r} \times \text{(momentum)} & \} &
 \boldsymbol{L} = \boldsymbol{r} \times \boldsymbol{p} & \} & L = d_\perp\, p }$$
$$\matrix{ \text{(moment of force)} = \boldsymbol{r} \times \text{(force)} & \} &
 \boldsymbol{\tau} = \boldsymbol{r} \times \boldsymbol{F} & \} & \tau = d_\perp\, F }$$
See also this answer for why cross products in physics related to the moment arm (minimum distance to line).
The truth of the matter is that angular momentum (and all other moments of xxx) only tell us where momentum is acting though. It is a supplementary quantity that describes the geometry of a problem. See this answer asking of torque is a fundamental quantity. Same argument applied to angular momentum.
When angular momentum is conserved, it means that not only the magnitude and direction of momentum is conserved, but also the line in space where momentum acts through. The location where momentum acts is commonly called the center of percussion, although it is actually an axis.
