Say there is a block sliding down an inclined plane that rests on a frictionless table. There is kinetic friction between the block and inclined plane. If the block slides downhill, then the kinetic friction acting on it points uphill. By Newton’s third law, the inclined plane will experience a friction force pointing down hill, in the direction of the block’s velocity/acceleration. Shouldn’t the plane want to move with the block, then? If it does, then wouldn’t the center of mass move with the block and plane too? There’s no friction on the block–plane system, however, so the center of mass should not move, but my analysis claims that it does. Where have I gone wrong?
The 2 forces acting on the wedge due to the block are the normal reaction $N$ and the friction force $F=\mu |N|$ (see diagram). Both have horizontal components : $N\sin\theta$ and $F\cos\theta$ respectively. The net horizontal force acting to the left on the wedge is $N\sin\theta-F\cos\theta=(\tan\theta -\mu)N\cos\theta$.
The condition for the block to start sliding down the incline is $\tan\theta \gt \mu$. So if the block slides to the right then the horizontal force on the wedge is always +ve to the left.
Note that in this situation the normal force is not $N=mg\cos\theta$. The wedge accelerates to the left away from the plane of contact with the block, so the normal force is reduced from this value.
Treat the plane and the block as a system
When you will treat the block and the inclined plane as a system you find that the only forces which are acting on the system are in the vertical direction. therefore the center of mass of the system will accelerate ( if it does ) in a vertical line only. also the net force vector in this case will point downwards so the center of mass will accelerate downwards.
or to simplify the center of mass won't be accelerated in the horizontal direction.
The inclined plane does want to move with block
The friction which acts on the inclined plane definitely opposes the relative motion between them but what you forgot to count I think is the normal form the smooth friction-less table. The table may be smooth but it can still exert a normal force on any body that presses it.