What is actually Gravitational potential? Gravitational potential is work done by gravitational force to bring unit mass to its field from infinity.
but how can gravitational force work there? I mean, in infinity? As it can only work at his own field! And where the gravitational force is inactive, we say it infinity.
 A: Physically, infinity is impossible. So when we talk about bringing in particles from infinity, we mean from a very far distance away where the gravitational field is negligible but still technically non-zero. Of course mathematically infinity is fine, and this is why it is spoken of in this way.
A: Remember that a potential $\Phi$, say in this case gravitational, is an arbitrary function which in physics follows the equations:
$\vec{F}=-\nabla\Phi$
$\Phi=-\int\,\vec{F}\cdot d\vec{r}$
whereas you talk about a conservative force field $\vec{F}$. Work is related to the second equation, BUT TYPICALLY (AND MATHEMATICALLY) it is a path integral. The potential (when evaluated between two points in the field) will be equal to minus the work exerted.
Also mathematically, $\Phi$ can be any function, so its physical importance or meaning is negligible (as you can introduce any function that respects the equation above). In other words, you don't measure a gravitational potential in a laboratory, but you can measure the action that the gravitational field exerts on a point mass - for instance - and then work out a potential for it, in case it is possible. In electrostatics, the same occurs for the electric field $\vec{E}$ and the electric potencial $V$.
I don't know if you are familiar with this, but let's see. Consider the Poisson equation for a gravitational potential:
$\nabla^2\Phi=4\pi G\rho$
which leads to famous Newton's gravitational law:
$\vec{F}=-\dfrac{GMm}{r^2}\hat{r}$
When you solve this partial differential equation for this problem, you get that the gravitational potential (by unit mass) is:
$\Phi_m=-\frac{GM}{r}$ with
$\rho=M\delta(r)$
where $\delta(r)$ is Dirac's delta function in spherical coordinates for the radius $r$. The latter shows that in $r=0$ you have the physical singularity of the "big mass $M$" object and due to it, you have a physical interaction for $r>0$.
The force vector field for this potential will be "active" for any point in its space, but in "infinity", which for us in physics means $r\rightarrow\infty$ but not equal (and you can check it directly from Newton's equation above), force will be very, very, very weak. 
A: As you said gravitational potential energy $U$ represents the potential some body has to do work located at some point in a gravitational field. For a uniform gravitational field we can approximate the gravitational potential energy (PE) as 
$$ U = \text{Work} \times \text{height} = mgh, $$ 
where $h$ is usually taken to be the height above the surface of the Earth. For this particular approximation of PE we normally assign the zero of potential energy at $h=0$. This is synonymous with deciding on the origin of a coordinate system. It makes sense. However, it is arbitrary and can be placed anywhere you like so long as you remain consistent. 
What about the more general potential 
$$ U = \frac{GMm}{r},$$
The choice of placing our zero point for the gravitational potential energy at $r=\infty$ is convenient for caluclations. It does make sense. As we start to achieve large $r$ values, the gravitational force is rapidly approaching $\infty$. So the further we get away from an object the gravitational force is rapidly decreasing. Or, in other words, the gravitational force quickly asymptotes to zero. So it makes sense to have the zero of the gravitational PE at $r=\infty$. 
