Is the labelling of $g_{\mu\nu}$ as "the metric" and $g^{\mu\nu}$ as "the inverse metric" arbitrary? I understand the up and down indices change the way in which the metric transforms under basis changes, that's not what I'm getting at. My question is that since the metric is specifically an isomorphism between a vector space $V$ and its dual $V^*$ is there any reason why $g_{\mu\nu}$ is the "regular" metric and $g^{\mu\nu}$ is the "inverse"? Or is this simply a matter of convention?
 A: Well, four-velocity $dx^\mu/d \tau$ lives on the tangent space $T \mathcal{M}$. This is actually how tangent space is defined, as the space of tangents to curves. In other words, this is where the whole story of tensors starts and all the other structure is built from its relation to $T \mathcal{M}$. So it is perhaps natural to define the metric first as acting on the "primary" tensors living on $T \mathcal{M}$, i.e. as a two-form $g_{\mu\nu}$.
Another, probably more convincing argument is that ultimately you would like the metric to measure the length of curves. In Riemannian geometry this is done as
$$\ell_\gamma \equiv \int_\gamma \sqrt{g_{\mu\nu} \frac{d \gamma^\mu}{d \lambda} \frac{d \gamma^\nu}{d \lambda}} d \lambda$$ 
for an arbitrarily parametrized curve $x^\mu=\gamma^\mu(\lambda)$. Once again, the primary object appearing there is $g_{\mu\nu}$ because it acts on curve tangents. It is the variation $\delta\ell_\gamma/\delta \gamma = 0$ that defines geodesics and (under the right conditions) the distance function on Riemannian spaces. Even though these arguments are a little bit more nuanced in relativity (e.g. there is no good distance function that can be defined with a pseudo-Riemannian metric), it is nice to keep the Riemannian analogy and call $g_{\mu\nu}$ "the" metric tensor and $g^{\mu\nu}$ its inverse.
A: The metric $\mathbf g: V\times V \rightarrow \mathbb R$ is, by definition, a multilinear map which eats two elements of the underlying vector space and spits out a real number.  
Because the metric provides an isomorphism$^\dagger$ between the vector space $V$ and the dual space $V^*$, we can map any vector $X\in V$ to a corresponding covector $X^\flat \equiv \mathbf g(X,\bullet) \in V^*$, and any covector $\omega\in V^*$ to a corresponding vector $\omega^\sharp \in V$.
This being the case, we can choose inherit a metric $\boldsymbol \Gamma$ on $V^*$ from the already-defined metric $\mathbf g$ on $V$ as follows:
$$\boldsymbol \Gamma(\omega,\sigma) := \mathbf g(\omega^\sharp,\sigma^\sharp)$$
In a mild abuse of terminology, we call $\boldsymbol \Gamma$ the "inverse metric" because the components $\Gamma^{ij}$ are found by taking the matrix inverse of the components $g_{ij}$. 

Once we settle on a fundamental vector space $V$, there is no ambiguity$^{\dagger \dagger}$ - the metric is a bilinear form on $V$, while the "inverse metric" is a bilinear form on $V^*$.  However, because $V$ and $V^*$ are isomorphic$^\dagger$, we could in principle start from $V^*$ as our fundamental vector space, and everything else would proceed with the labels reversed.  In that sense, the distinction is only conventional.

$^\dagger$This is only true if $V$ is finite-dimensional.  If $V$ is an infinite dimensional vector space, then $V$ and $V^*$ are no longer isomorphic.
$^{\dagger \dagger}$ Unless you want to redefine the metric to be a bilinear form on the dual space, but at this point we're just redefining words, and if we're doing that then everything is conventional
