Is there any physics behind covariance and contravariance of indices of tensors? Is there any physics behind covariance and contravariance (up and down) of indices of tensors?
 A: In one sense, no.  It is simply geometrical in nature.  The whole of (pesudo-)Riemannian geometry is pure mathematics.  You can construct the Ricci, Riemann, and Einstein tensors totally through mathematics describing curvature.  In this sense, an example of a physical statement being made is $G_{ab} = 8\pi T_{ab}$, equating the Einstein tensor with the stress-energy tensor.  It's an equation that cannot be arrived at entirely through mathematical symbol pushing; it requires experimental evidence to justify it.
Now on the other hand, you could argue that we needed to do experiments to determine what kind of geometrical world we live in.  That is true, but to me, the concepts of contravariance and covariance exist regardless of whether they are present in the physical world.  It is a physical statement to say that physical quantities transform according to those laws--which is part of a larger statement that we live in a pseudo-Riemannian manifold and that that geometrical framework is applicable to our world.
A: A coordinate like $V^\mu$ is a coordinate of a vector, a vector is a element of the tangent bundle.
A coordinage like $F_\mu$ is a coordinate of a $1$-form, a $1$-form is a element of the co-tangent bundle.
The co-tangent bundle  could be seen as a "dual" of the tangent bundle, that is $<f^\mu,v_\nu> = \delta^\mu_\nu$, where $f$ is a basis for $1-$form, and $v$ is a basis for vectors.
Now, locally, the metrics $g_{\mu\nu}$ establish a correspondance between the tangent  bundle and the co-tangent bundle :  $A_\mu = g_{\mu\nu} A^\nu$ 
This means for instance, that the "potential vector" $A_\mu$ is not a coordinate of a vector, it is a coordinate of a $1$-form  $A = A_\mu dx^\mu$, while, locally, you could find an equivalent vector thanks to the metrics, but it works only locally. 
So, covariant and contravariant notations help us, to correctly understand the structure of physical quantities.
