While doing parallel transport, what are we actually doing? The Covariant Derivative which is used in General Relativity is obtained by considering the concept of Parallel transport of a vector. 
A vector is transported parallelly from one point on the manifold to the other because we cannot do algebraic operation on objects at two different points on a manifold. 
My question is while doing a parallel transport what are we doing actually? What is the action of Parallel transport means mathematically? Mathematically, what operation on a vector makes to go under a parallel transport? 
 A: A vector is a geometric object defined by the components and a coordinate basis. On a curved manifold both the components and the basis change from one point to the other. The partial derivative is not enough to describe the change of the vector as an object; hence the need to define the covariant derivative, which measures the change of a vector on a curved spacetime in a way independent of the coordinates. Technically it is built on the partial derivative plus a correction term, that is the connection, to make the operator a tensor. Mathematically the covariant derivative takes count of the change in both the components and the coordinate basis, as you move along a path on the manifold. In the parallel transport you apply the covariant derivative along the path and requires it vanishes. As the operator by definition measures the change of the geometric object, the vanishing assures it remains parallel.
A: The geometric intuition is that parallel transport over a curved surface is different from that on a flat surface. 
Draw a triangle on a flat surface and parallel transport a vector along it; when you return to your starting point you will find that it's the same vector. 
Now draw a triangle on a sphere starting from the North Pole then going to the equator and then going along the equator a quarter turn and then returning to the North Pole. Now if you parallel transport a vector from the North Pole along this triangle you will find that when you return to the North Pole that the vector has changed its orientation. 
Of course, the above is is visual aid and not a precise way of encoding what is meant by parallel transport. Mathematically, the notion is precisely encoded in concepts like connection, holonomy, covariant derivative and parallel transport on manifolds or more generally in vector or principal bundles. 
