Now the default answer to this may be, "It has no origin because it's a definition", but let me just try to justify my concern here.
On page 842, equation (22.91) of "Modern Electrodynamics" by Andrew ZangWill, 2013, Cambridge university press. It is written that "$[\cdots]$ if we can define the four-vector $A^\mu=\left(\vec A,\, i \phi/c\right)$". While on Wikipedia it is simply stated that "The contravariant electromagnetic four-potential can be defined as $A^\mu=\left(\phi, \vec A\right)$", on that page there is a citation to the text by D.J. Griffiths (2007). "Introduction to Electrodynamics (3rd ed.)". I have that book and what he actually writes is $A^\mu=\left(V/c,A_x,A_y,A_z\right)$ on page 541, eqn. (12.131), which is fine and a subtlety I won't dwell over. But I would like to emphasize the words in bold here which seem to suggest that this 'definition' is not unique.
Now you may wonder why all the concern regarding $A^\mu$ and why not fixate on some other four-vector, like the four-current (density) $j_\mu=\left(-\rho,\vec J\right)$ for instance.
The reason I'm okay with that equation is because the four-current has an origin, unlike $A^\mu$, the four-current originates from the sourced Maxwell's equations $$\begin{pmatrix}\nabla\cdot \vec E \\\\\frac{\partial{\vec E}}{\partial t}-\nabla \times \vec B\end{pmatrix}=-\mu_0 \begin{pmatrix}-\rho\\\\\vec J\end{pmatrix}\tag{1}$$ Where I identify the vector on the RHS of $(1)$ as $j_\mu=\left(-\rho,\vec J\right)$ or the contravariant form as $j^\mu=\left(\rho,\vec J\right)$ - by raising the zeroth component with the inverse Minkowski metric for which the convention I used is $\eta=\mathrm{diag}(-1,1,1,1)$. Eqn. $(1)$ was written in natural units ($c=1$).
Equation $(1)$ tells me that 'sourced' Maxwell equations lead to the zeroth component being a scalar, $\rho$ and the other 3 components combine to a 3-vector, $\vec J$. This is understood and makes sense to me as $\nabla\cdot \vec E=\mu_0 \rho$ is a scalar equation and $\frac{\partial{\vec E}}{\partial t}-\nabla \times \vec B=-\mu_0 \vec J$ is a 3-vector equation.
Most textbooks normally just state $A^\mu=\left(\phi, \vec A\right)$ without any explanation as to why it takes that form. By 'form' here, I mean why does it have a scalar, $\phi$ component, and a 3-vector component, $\vec A$. Why not $A^\mu=\left(\phi_1,\phi_2, A_1, A_2\right)$ for instance? Also why is there not some other choice of sign, say, $A^\mu=\left(-\phi, \vec A\right)$ or $A^\mu=\left(\phi, -\vec A\right)$ or even $A^\mu=\left(-\phi, -\vec A\right)$?
I know there may not be a simple answer to this question, but to summarize, can anyone please explain to me where the four-vector potential originates?