Let's say I stay at point $x=0$ in vacuum. When I move my arm forward such that it will have a positive $x$ position (say $x=5$) will the rest of my body move backward such that it will have a negative $x$ position (like $x=-1$)?
There are only internal forces of your body. Without external forces, the center of mass of your body cannot change position. As your center of mass did not move, the main body should move in the opposite direction.