But when the force is applied at any other point on the rod, say on one of its ends, the center of mass will still accelerate with the same acceleration
It will not have the same translational acceleration (increase in translational kinetic energy) when the force is applied to the end as when the force is applied to the COM, for a given displacement of the point of application of the force. That’s because the displacement of the COM will be less when the force is applied to the end than when applied to the COM. See the figures below.
For both figures the work done by the force $F$ is the product of the the force and the displacement of its point of application. For both figures that work is $Fd$. Note however in the figure to the right the displacement $d_{COM}$ of the COM due to the force is less than the displacement $d$ of the point of application of the force. Therefore the increase in translational kinetic energy of the COM is less when the force is applied to the end than when it is applied to the COM for the same displacement of the point of application of the force.
Since the net work done on an object equals its total increase in kinetic energy, the decrease in translational kinetic energy when the force is moved from the COM to the end equals the rotational kinetic energy resulting from moving the force.
Hope this helps.