In mechanics, we love energy: indeed, it often really simplifies solving problems. Often, we want to know how the speed of a particle varies, or, in other words, what its kinetic energy is. Luckily, for some forces, knowing the variation in kinetic energy is really easy: these forces are the conservative forces. A force $\textbf{F}$ is conservative when its work along a path $\Gamma$ only depends on the initial and final positions: then, we can introduce a potential energy, say $E_p$, so that
$$ W(\textbf{F}) = \int_\Gamma \textbf{F}\cdot\textbf{dl} = E_{pi} - E_{pf}$$
However, the kinetic energy theorem states that the total work done on a particle equals its kinetic energy variation, so
$$ \Delta KE = W(\textrm{F}) = -\Delta E_p $$
Now, let's focus on Coulomb's law. Consider a charge $Q$ in $P$, and another one, say $q$ in $M$. It appears that $Q$ creates a force $\textbf{F}_e$ on $q$, with
$$\textbf{F}_e = \frac{Qq}{4\pi\epsilon \|\textbf{PM}\|^3}\textbf{PM}$$
Since $\textbf{F}_e$ only depends on the position of $P$ and $M$, it is derived from a potential energy $E_p$, and we find that
$$ E_p = \frac{Qq}{4\pi\epsilon\|\textbf{PM}\|}$$
However, even if the notation is usefull for a few particles, it really becomes difficul when studying a complex distribution of charges. Then, we prefer to see a single charge as the source of an electric field $\textbf{E}$, which exists everywhere, with $$\textbf{E}(M) = \frac{Q}{4\pi\epsilon \|\textbf{PM}\|^3}\textbf{PM}$$
Now, multiplying $\textbf{E}$ by the charge $q$ gives the electric force created by $Q$. So, $\textbf{E}$ and $\textbf{F}_e$ are almost the same: we introduce $U$ which is the potential created by $Q$, so that $E_p = qU$.