Conventionally, the velocity-dependent potential energy is $$U~=~q(\phi \dot{x}^0 -\vec{A}\cdot \dot{\vec{x}})~=~\mp q A_{\mu}\dot{x}^{\mu} ,$$
where
$$A^{\mu} ~=~(\phi,\vec{A} ), \qquad A_{\mu} ~=~(\mp \phi,\pm \vec{A} ), $$
in Minkowski signature $(\mp,\pm,\pm,\pm)$, respectively. (Here we work in units where $c=1$.)
Note in particular that the electric potential $\phi$ always comes with a plus sign in the velocity-dependent potential energy $U$ (when the charge $q$ is positive).