Your intuition is correct. All the points along the axis of rotation share the same motion. In fact, defining the center of mass by means of the rotation axis isn't sufficient. 

It is possible to define the center of mass as the only point where _all possible rotation axis pass through_ for a freely rotating rigid body. So you have to consider all possible lines passing through the center of mass (called a pencil of lines) to define the center of mass. 

This is a consequence of Newton's second law and the concept of momentum. The definition of momentum for a collection of particles leads to the definition of center of mass as being the special point whose motion and combined mass can be used to describe all the individual contributions of momentum from each particle.

$$ \vec{p} = \sum_i m_i \vec{v}_i = \left( \sum_i m_i \right) \vec{v}_{\rm COM} = m \, \vec{v}_{\rm COM} $$

There is only one point on a rigid body that satisfies the above. And the condition for this point is the definition for center of mass

$$ \sum_i m_i \vec{r}_i = \left( \sum_i m_i \right) \vec{r}_{\rm COM}  $$ or $$ \vec{r}_{\rm COM} = \frac{\sum_i m_i \vec{r}_i}{\sum_i m_i} $$

PS. In geometry, a point is equivalent to a pencil of lines through that point. Just like a line is equivalent to a locus of points along the line.

PS2. See also [this similar answer](https://physics.stackexchange.com/a/568461/392) to a related question.