I'm having a little trouble understanding what makes a planetary orbit explicitly elliptical. Is it simply that the initial velocity was different from the circular orbit case for a given starting position (radius from the sun) and this becomes an elliptical orbit (or more generally, an arbitrary conic section)?
Is there any other information contained in the eccentricity of an orbit?
There is in fact more information contained in the eccentricity. The Kepler problem (in fact, every two body problem with an inverse-square force) has a subtle symmetry: It is symmetric under certain rotations of the four dimensional space. By Noether's theorem, every symmetry of a system corresponds to a conserved quantity. One can show that the vector $$\mathbf{A} = \mathbf{p} \times \mathbf{L} -mK\hat{r}$$ $\mathbf{A}$ is called Laplace-Runge-Lenz-vector. Calculating the scalar product of $\mathbf{r}$ and $\mathbf{A}$ yields $$\mathbf{r} \cdot \mathbf{A} = A r \cos\phi = \mathbf{r}\cdot(\mathbf{p} \times \mathbf{L}) - mKr = \mathbf{L}\cdot (\mathbf{r} \times \mathbf{p}) - mKr = L^2 - mKr.$$ Solving the above equation for $r$ and some algebraic manipulations give us $$r = \frac{\frac{L^2}{mK}}{1 + A\frac{\cos\phi}{mK}}.$$ As you can see, this is the equation of the orbit of a body under an inverse square where the eccentricity $\epsilon$ is related to the magnitude of the LRL-vector: $$\epsilon \equiv \frac{A}{mK}.$$ The fact that the orbits are conic sections is directly related to a conserved quantity and hence to a symmetry of the system.
The orbit (in polar coordinates) of a body under a inverse-square force, $-K/r^2$, is given by $$r(\phi)=\frac{L^2/mK}{1+\sqrt{1+\frac{2L^2E}{mK^2}}\cos\phi},$$ where $E$ and $L$ are the energy and the angular momentum of the particle. The equation above is just the polar representation of a conic section of conic parameter $L^2/mK$ and eccentricity $$\epsilon=\sqrt{1+\frac{2L^2E}{mK^2}}.$$
It is clear since $$E=\frac{m\dot r^2}{2}+\frac{L^2}{2mr^2}-\frac{K}{r},$$ and $$\vec L=\vec r\times\vec p,$$ that the eccentricity depends upont the upont the initial velocity $\vec v(0)$ and the initial distance to the origin $r(0)$. Knowing these quantities you can compute the eccentricity of the orbit.
