Although the links in the comment sections are recommended for those interested, I will attempt to provide more of a heuristic argument for the Levi-Civita connection. As the OP states the Levi-Civita connection is uniquely defined by requiring metric compatability and zero torsion:
$g_{ij;c} = 0$
$v^i{}_{;j}u^j - u^i{}_{;j}v^j = [u,v]$
where $[u,v]$ is the commutator (Lie bracket) of the two vectors $u$ and $v$. Let us begin with the first demand.
1: Physically there is no difference between vectors and co-vectors; although they arise from different mathematical contexts both simply describe directions in spacetime, and any one direction ought to be the same whether it is defined by a vector or a co-vector. Therefore it is physically reasonable to demand that the covariant derivatives of a vector $v^i$ and a co-vector $u_i$ define the same direction (vector/co-vector) whenever $v^i$ and $u_i$ do.
The identification of vectors and co-vectors can be carried out using the canonical isometry induced by the metric tensor, which is commonly described by the process of index raising/lowering:
$$
v^i \sim u_i \quad\text{if and only if }\quad v^i = g^{ij}u_j.
$$
We write the co-vector $u_i$ satisfying $u_i \sim v^i$ as $v_i$. The final piece of our argument is to let the connection on the tangent bundle, $TM$, induce a connection on the co-tangent bundle, $T^*M$. This is done by demanding that the connection acts naturally with respect to contraction:
$$
(v^iu_i)_{;j} = v^i{}_{;j}u_i + v^iu_{i;j}.
$$
Combining the isometry and the induced connection, we find
\begin{align}
u_{i;j}v^j = \left(g_{ik}u^k\right)_{;j}v^j = g_{ik;j}u^kv^j + g_{ik}u^k{}_{;j}v^j,
\end{align}
but since we, as argued, would like to demand that $u^k{}_{;j}v^j \sim u_{k;j}v^j$ we let the metric contraction in the last term lower the index:
$$
u_{i;j}v^j = g_{ik;j}u^kv^j + u_{i;j}v^j.
$$
Thus clearly $g_{ik;j}u^kv^j = 0$, and since this must hold for arbitrary vectors $u^k$ and $v^j$ we must have $g_{ik;j} \equiv 0$.
2: The matter of torsion is more complicated, whence the interest in Einstein-Cartan theory. As can be seen in the links provided in the comment section, non-zero torsion certainly would have measurable effects. However, since we would like to present a heuristic argument why we would initially choose to demand zero torsion, I can think of no better way than to compare the covariant derivative and the Lie derivative.
Consider therefore an group action on our manifold, such that a point $p$ is carried into a point $q$:
$$
x^\mu_p \mapsto x^\mu_q = x_p^\mu + \lambda v_p^\mu,
$$
where $x_p^\mu$ denote the coordinate functions at $p$ and $v^\mu_p$ is a vector at $p$, and $\lambda$ is very small. Ultimately, we will let $\lambda \to 0$ as we consider an infinitesimal group action. Then the Jacobian of this transformation, $J^\mu_\nu$, is given by
$$
J^\mu_\nu = \delta^\mu_\nu + \lambda v^\mu{}_{,\nu},
$$
so a vector $u^\mu$ transforms as
$$
u^\mu\rvert_p \mapsto u^\mu\rvert_q = \widetilde{u}^\mu\rvert_p + \lambda v^\mu{}_{,\nu}\widetilde{u}^\nu\rvert_p,
$$
where $\widetilde{u}^\mu$ is the vector before transformation. Thus we find
$$
u^\mu - \widetilde{u}^\mu = u^\mu\rvert_{x_p^\nu + \lambda v^\nu_p} - u^\mu\rvert_{x_p^\nu} - \lambda v^\mu{}_{,\sigma}u^\sigma\rvert_{x_p^\nu},
$$
so that
$$
\lim_{\lambda \to 0} \frac{u^\mu - \widetilde{u}^\mu}{\lambda} = u^\mu{}_{,\sigma}v^\sigma - v^\mu{}_{,\sigma}u^\sigma = [v,u].
$$
The first term in the middle expression comes from the difference between the untransformed vector at $q$ to that at $p$. If the group action can be considered "physical" in some sense, then perhaps we can agree that the result should be the covariant derivative of $u_\mu$ along $v^\sigma$:
$$
u^\mu{}_{,\sigma}v^\sigma \mapsto u^\mu{}_{;\sigma}v^\sigma.\tag{1}
$$
The second term, on the other hand, tells us how our transformation changes along $u^\sigma$. Again, if the transformation can be considered "physical" perhaps we can agree that the result should be the covariant derivative
$$
v^\mu{}_{,\sigma}u^\sigma \mapsto v^\mu{}_{;\sigma}u^\sigma.\tag{2}
$$
What do we mean by a "physical" transformation above? If you carry with yourself a set of rulers, which define vectors, then the change from if you parallel transported them is given by the covariant derivative as above. This represents $(1)$. On the other hand, these rulers, being physical objects, are themselves travelling through spacetime, and the difference between their motion and your motion is also given by the covariant derivative, in the infinitesimal case. This represents $(2)$. That is to say, we consider the change between the original vector defined by the ruler, and the vector defined by the transported vector compensating for the physical dimensions of our "vector"/ruler.
You may be familiar with this as the Lie derivative of a vector field, which is a priori not a covariant expression. However, if one accepts the above as an argument why it should be, we can achieve this by exchanging the partial derivatives for covariant derivatives, as per the prescription of the equivalence principle, which yields exactly zero torsion.
I am not certain that I have managed to present the argument very well, and it is clearly not water tight either way. It might also be relevant to consider this side of non-zero torsion.
