What causes the orbital eccentricity? I found some websites trying to attribute this to the external other forces affecting the 2 body problem to follow an elliptical rather than a circular orbit, but I am not convinced though. I think any 2 body problem can have an elliptical orbit (isolated from other external forces) as this is the general solution of Kepler's first law if derived from Newton's law of universal gravitation. Can we for example say that initial formation conditions are the main cause? 
 A: As you have said, the general solution for the Kepler problem is an elliptic orbit. The shape of an ellipse is determined by its semi-major axis and the eccentricity. The semi-major axis is determined entirely by the energy, but the eccentricity depends also on the angular momentum. Both energy and angular momentum are conserved in the two-body problem, so it is absolutely correct to say that the eccentricity comes from the initial conditions.
You can go a step further and say also that the orientation of the orbit in space is constant. Naturally perturbations from the other planets, tidal forces and in the case of Mercury effects from general relativity will change both the shape and orientation of orbits over time. 
A: In a two body system the orbit can have any eccentricity you want. The key point is that the eccentricity cannot change. In a three or many body system the bodies perturb each others orbits and the eccentricity can change with time.
For example this graph shows the eccentricities of the rocky planets as a function of time. These changes are mainly caused by Jupiter and Saturn (because they're the heaviest planets) - there is more info in the Wikipedia article on Milankovitch cycles.
We don't know what Earth's orbit was like when the Solar System first formed. There are suggestions that the orbits of planets can change radically as the system forms, though what implications this has for the Solar System aren't known. It is likely that any planets in wildly eccentric orbits would have been ejected over the lifetime of the Solar System, so the planets left behind would be the ones in approximately circular orbits. We also expect tidal forces from the Sun to gradually circularise planetary orbits.
