Components of 4-Potential and 4-Current in terms of E and B

So this question might be trivial but say we have a 4-vector $A^\mu=(\phi,\vec{A})=(\phi,A_x,A_y,A_z)$ and a 4-current $J^\mu=(J^0,\vec{J})$. We know that

$$\vec{E}=-\nabla\phi-\frac{\partial\vec{A}}{\partial t}$$
$$\vec{B}=\nabla\times\vec{A}$$

How can I put the components of $\vec{A}$ and $\vec{J}$ in terms of $\vec{E},\vec{B}$? For example, say I have $$J^\mu A_\mu=J^0\phi-J^1A^1-J^2A^2-J^3A^3.$$ How can I express this in terms of the electric and magnetic fields?

The first thing you have to do is pick a gauge, because the 4-potential isn't gauge invariant and will take different forms depending on which gauge you choose. The most commonly used are the Coulomb gauge and Lorenz gauge, but other possibilities exist. The Wikipedia article I linked to includes formulae for calculating the vector potential in the Coulomb gauge, and fully writing them in terms of $\vec{E}$ and $\vec{B}$ is just a matter of using the Maxwell's equation $\nabla \cdot \vec{E} = -\frac{\rho}{\epsilon_0}$ to get \begin{align} \Phi\left(\vec{r},t\right) & = -\frac{1}{4\pi} \int \frac{\nabla\cdot \vec{E}\left(\vec{r}',t\right)}{\left|\vec{r}-\vec{r}'\right|}\operatorname{d}^3r',\ \mathrm{and} \\ \vec{A}\left(\vec{r},t\right) & = \nabla\times\int \frac{\vec{B}\left(\vec{r}',t\right)}{4\pi \left|\vec{r}-\vec{r}'\right|}\operatorname{d}^3r'. \end{align}
Finding $J^\mu$ from $\vec{E}$ and $\vec{B}$ a straightforward application of Maxwell's equations.