Yes, you are right. $qA$ is called the field momentum. And it is indeed equal to the momentum carried by the electromagnetic field. It can be derived from the definition of Poynting vector, where you try to express its contribution to the total electromagnetic field energy.
Actually, it can be expressed using a wide variety of expressions, given by
$$\mathbf{P_{EM}= qA = \int_V \rho A \, dV = \epsilon \int (E \times B) \, dV}$$$$\boxed{\mathbf{P_{EM}= qA = \int_V \rho A \, dV = \epsilon \int (E \times B) \, dV}}$$
You may check this article for details.
Also, asAs mentioned by @ChiralAnomaly in the comments, this assumes the validity of the Coulomb gauge $\mathbf{\nabla \cdot A =0}$. This This renders the vector potential to be a bit unrealistic but it is good enough for semi-classical calculations. In general, $\mathbf{\int_V \rho A \, dV = \epsilon \int (E \times B) - E \,(\nabla \cdot {A}) \, dV}$.