Let's say we are talking about gravitational potential energy, it is defined as:
At a point $X$, inside the gravitational field of mass $M$, the work done by an external agent to bring a mass $m$ from infinity to that point $X$ without changing its kinetic energy,is called the potential energy of that system.
Why don't we define it this way? We know,for conservative forces..like gravitational force, $W_{c}=-\Delta{U}$
$\implies$ $\Delta{U}=-W_c$
My question is why do we always define potential energy in terms of external forces?Isn't defining it in terms of conservative forces much simpler?