No, it would not because conservative forces, by definition depend only on the position. Your force should depend also on the velocity (its direction) because it is perpendicular to it.
Forces whose work is always zero are not necessarily positional so that they are not necessarily conservative. There are two important cases in elementary physics: Coriolis' force and magnetic force.
For these forces no potential energy can be defined so that non energy conservation theorem can be formulated
(where these forces give some contributiin).
In more advanced formulations of mechanics a generalized notion of potential can be introduced that depends also on the velocities, but there is not necessarily a theorem of total energy conservation (there is if further hypotheses are true and is called Jacobi's theorem).