Your intuition is correct. All the points along the axis of rotation share the same motion. In fact, defining the center of mass by means of the rotation axis isn't sufficient.
It is possible to define the center of mass as the only point where all possible rotation axis pass through for a freely rotating rigid body. So you have to consider all possible lines passing through the center of mass (called a pencil of lines) to define the center of mass.
This is a consequence of Newton's second law and the concept of momentum. The definition of momentum for a collection of particles leads to the definition of center of mass as being the special point whose motion and combined mass can be used to describe all the individual contributions of momentum from each particle.
$$ \vec{p} = \sum_i m_i \vec{v}_i = \left( \sum_i m_i \right) \vec{v}_{\rm COM} = m \, \vec{v}_{\rm COM} $$
There is only one point on a rigid body that satisfies the above. And the condition for this point is the definition for center of mass
$$ \sum_i m_i \vec{r}_i = \left( \sum_i m_i \right) \vec{r}_{\rm COM} $$ or $$ \vec{r}_{\rm COM} = \frac{\sum_i m_i \vec{r}_i}{\sum_i m_i} $$
PS. In geometry, a point is equivalent to a pencil of lines through that point. Just like a line is equivalent to a locus of points along the line.
PS2. See also this similar answer to a related question.
PS3. Here is an overview of the development of equations of motion for a rigid body. This might be advanced reading at this point, but it contains all the concepts needed to understand the subject.