"Contraction-orthogonality" of covariant and contravariant basis
Contravariant vectors or just "vectors" are defined as elements of the tangent space at a given point. In practice, they are defined with respect to a coordinate-vector basis $\mathbf{e}_{(i)}$, where $\mathbf{e}_{(i)}$ is the vector tangent to the $i$-th coordinate line. Then they are given, as per usual, as a linear combination of the basis vectors (Einstein summation assumed henceforth)
$$\mathbf{v} = v^i \mathbf{e}_{(i)}$$
Covariant vectors or "1-forms", on the other hand, are more abstract objects which are defined exclusively through their action on contravariant vectors. For instance, a 1-form acting on a (contravariant vector) will return a number
$$\mathbf{\alpha}(\mathbf{v}) = C,\,C \in \mathbb{R}$$
Another requirement on the 1-form $\alpha$ is that its action is linear, that is for any vectors $\mathbf{v},\,\mathbf{w}$ and any two constant $D$, $E$ $$\alpha(D \mathbf{v} + E \mathbf{w}) = D \alpha(\mathbf{v}) + E \alpha(\mathbf{w})$$
This actually means we can completely reconstruct its action from the components
$\alpha_i \equiv \alpha(\mathbf{e}_{(i)})$ because then thanks to the linearity (verify this yourself)
$$\alpha(\mathbf{v}) = \alpha_i v^i$$
Alternatively, you can define $\alpha_i$ as components with respect to a covariant basis
$\alpha = \alpha_i \epsilon^{(i)}$ where $\epsilon^{(i)}$ is defined by the property
$$\epsilon^{(i)}(\mathbf{e}_{(j)}) = \delta^i_j$$
That is, the "contraction-orthogonality" of covariant and contravariant bases has nothing to do with a geometrical definition of distance or angles (AKA the metric).
Raising and lowering indices
Now consider a space where we actually have a metric $g_{ij}$, abstractly we can define it as
$$\mathbf{g}(\mathbf{v},\mathbf{w})=C,\,C \in \mathbb{R}$$
where we again require linearity. Now consider the form $\kappa$ defined by
$\kappa(\mathbf{w}) \equiv \mathbf{g}(\mathbf{u},\mathbf{w})$
for some vector $\mathbf{u}$. $\kappa$ is a full-fledged form as defined above, its components can be shown to be
$$\kappa_i = g_{ij} u^j$$
physicist know this as "index lowering". We assume that the metric is non-degenerate which means that $u \leftrightarrow \kappa$ is a one-to-one relation and we can find an inverse $\mathbf{g}^{-1}$. The components of the inverse metric are usually denoted as $g^{ij}$ where $g_{ij} g^{jk} = \delta^i_k$ is a defining property. The "index-raising" from forms to vectors is then defined by
$$w^i = \beta_j g^{ij}$$
In fact, this operation is so common in metric geometry that physicists (especially relativists) simply say that it is "the same object with indices up or down", i.e.
$$\kappa_i = v^j g_{ij} \equiv v_i$$
and
$$w^i = \beta_j g^{ij} \equiv \beta^i$$
Non-orthogonal bases
We can take the contravariant basis vectors and check whether they are orthogonal by taking simply
$$\mathbf{g}(\mathbf{e}_{(i)},\mathbf{e}_{(j)}) =?$$
or the covariant basis
$$\mathbf{g}^{-1}(\mathbf{\epsilon}^{(i)},\mathbf{\epsilon}^{(j)}) =?$$
In this sense, covariant and contravariant basis vectors can be orthogonal or non-orthogonal.
I am slightly uncomfortable with the way the article above "mixes" the covariant and contravariant vectors together. Trying to assess whether $\mathbf{e}_{(i)}$ and $\epsilon^{(j)}$ are orthogonal does not make a lot of sense because they are simply different geometrical objects. (If you try to raise/lower an index on either of them via the metric and then using the metric to dot-product the result, you will get orthogonality trivially from definitory properties of the bases and the metric/inverse metric cited above. You can easily verify that yourself.)
