What exactly is moment? Why does it correspond to a rotation? Wikipedia says:

Moment is an expression involving the product of a distance and physical quantity.

I don't quite get it. Moment is a vector, the cross product of the distance vector and the vector of the physical quantity, and it is linear. Why is it defined to correspond to an angular quantity like:
The moment of force is torque, and it is the product of angular acceleration and the rotational inertia;
The moment of momentum is angular momentum, the product of angular velocity and rotational inertia.
Edit number 1: I understand the definition. But why does the moment vector have to be constrained lie on the axis?
 A: I will ask one question: what about the moment of inertia?
I think understanding why such a way of defining moments is useful is much more important than investigating the definition of a moment itself.
If you have a solid example of a certain concept, it's much easier to extend that to a more general case and find a comprehensive reasoning. So I will take the moment of inertia as an example, as well as the moment of force (torque).
The moment of inertia is defined as: $$I = \int dm  \ r^2$$ where $r$ is the distance from the rotational axis and $dm$ is a mass infinitesimal or, mathematically, the integrating variable.
So, the first thing you may notice is that $I$ is a scalar quantity. The moment does not have to be a vector quantity. That's one thing. As the Wikipedia page states, the moment can be either vector or scalar, and we can call anything in the form $r^n\times appropriate \ physical \ quantity$. Perhaps you can think of $r$ as being the magnitude of the vector $\vec{r}$ so that we can see the identicality of the scalar moment and the vector moment.
I will now move onto your second question: what determines the direction of the moment vector, for example the torque (the moment of force)? As you can see from its formula $\vec{\tau}=\vec{r} \times \vec{F}$, the operation that relates the radius (moment arm) vector and the force vector is cross product. If the torque were to be defined as $\tau = \vec{r} \cdot \vec{F}$ using the dot product (which doesn't make any physical sense at all) then the torque would have been a scalar, not a vector. So the fact that the torque vector is orthogonal to both the radius vector and the force vector precisely comes from the pure mathematical properties of the vector cross product operation. There's nothing to do with the physics here, other than that we can think the direction of the torque vector as being the direction of the rotational axis.
I will finish by leaving another comment on how we determine the positive/negative directions of torque vectors, which is namely the right hand rule. The right hand rule is followed not because it is a mathematical truth, but because mathematicians and physicists all over the world just chose to set that as a standard convention.
A: To Edit number 1 : Well, if our world is 2 dimensional (only x, y axis), moment need not be a vector. If you rotate counterclockwise(ccw), you can just assign a positive scalar number, and if clockwise(cw), you can assign a negative scalar number.
One good fact of the 3 dimensional rotation is we can always define a single axis of rotation (this might look like a trivial statement in any dimension, but actually it is not). Therefore, we can consistently use our 2 dimensional case example, by assigning CCW to vector positively aligned to that axis, and for CW negatively aligned.
It is just a way to describe the direction and size in a consistent way. If you need more detailed treatment, google 'pseudovector' or 'axial vector'.
