Is the distinction between covariant and contravariant objects purely for the convenience of mathematical manipulation?

Two kinds of indices, covariant and contravariant, are introduced in special relativity. This, as far as I understand, is solely for mathematical luxury, i.e. write expressions in a concise, self-explanatory notation. For example, instead of writing the metric as $(\Delta s)^2=c^2(\Delta t)^2-(\Delta \textbf{r})^2$ one can write $x^\mu x_\mu$ which is not only a compact notation but also tells us that this expression is Lorentz invariant. But both $x_\mu$ and $x^\mu$, represent same objects: a set of four co-ordinates $(ct,x,y,z)$.

In the case of representations of $\mathrm{SU}(N)$, there too appear objects such as $\psi^i$ and $\psi_i$ which transform differently but keep $\psi_i\psi^i$ invariant. But we see that two different kind of objects exist in nature: quarks and anti-quarks which belong to the representations $\psi^i$ and $\psi_i$ respectively.

Does it mean in the latter case the distinction between covariant $\psi_i$ and contravariant $\psi^i$ is more fundamental than in the former case?

You need to be much more careful what the "upper" and "lower" indices denote and where they originate from. I'll discuss the two different "types" of upper/lower indices you are talking about:

Tensor indices

The first source of "objects with indices" is differential geometry. On any coordinate patch $U\subset M$ of a manifold $M$ with coordinates $q : U \to \mathbb{R}^n$, the coordinates themselves are traditionally written with "upper" indices $q^i$. On the manifold, there are now two closely related, yet different objects we naturally want to consider: Vector fields and differential forms. One way to define the tangent space at a point $q_0\in q(U)$ (corresponding to a point $p\in U$ as $q(p) = q_0$ is as the vector space spanned by the derivatives $\partial_i := \frac{\partial}{\partial q^i}\rvert_{q = q_0}$, whose indices are traditionally placed below. The cotangent space is the dual vector space spanned by the dual basis $\mathrm{d}q^i$ defined by $\mathrm{d}q^i (\partial_j) = \delta^i_j$.

Now, given any vector field $V$, we can expand it in the basis as $V = v^i(q)\partial_i$ for functions $v^i$, where the summation convention is in effect, i.e. we sum over all possible values of $i$. It is the $v^i(q)$ which is what a physicist refers to as "vector". Under a coordinate change, these components transform by the Jacobian matrix of the coordinate transformation. Conversely, we can expand a differential form as $\omega = \omega_i(q) \mathrm{d}q^i$, and it is the $\omega^i$ which the physicist usually calls "the form". These transforms by the inverse Jacobian matrix. vectors and covectors, and likewise differential forms and vector fields are, a priori, completely different things and should be conceived of as distinct geometrical concepts.

However, the waters are muddled because in physics we are often on a (pseudo-)Riemannian manifold with a metric tensor $g$ that defines the so-called musical isomorphisms between vectors and covectors by associating the 1-form $g(v,-)$ to a vector field $v$. Once in this setting, we can freely change the type of tensors and the originally distinct concepts become fully equivalent and interchangable in practical computations.

At this point, I would like to take issue with a certain part of the question:

Two kinds of indices, covariant and contravariant, are introduced in special relativity. This is as far as I understand, is solely for mathematical luxury i.e., write expressions in a concise, self-explanatory notation. For example, instead of writing the metric as $(Δs)^2=c^2(Δt)^2−(Δr)^2$ one can write $x^μx_μ$ which is not only a compact notation but also tells us that this expression is Lorentz invariant. But both $x^μ$ and $x_μ$, represent same objects: a set of four co-ordinates $(ct,x,y,z)$.

Although very close to the usage in practice, this is formally just non-sensical, precisely because the geometric objects are not considered properly. If $x^\mu$ is a set of coordinates, then there is no such thing as $x_\mu$ - you cannot lower the index of a coordinate because it is not a vector or tensor field, and therefore the musical isomorphism is not defined on it. The metric tensor encoded in $\mathrm{d} s^2$ (or $\Delta s$, as the question writes) does not act on coordinates, it acts on tangent vectors. The "distance" between two points is given by the extremum of the functional $$L[\gamma] = \int_\gamma \sqrt{g(\dot{\gamma},\dot{\gamma})}\mathrm{d}\tau$$ for paths $\gamma$ between the two points. Since the shortest lines, i.e. geodesics, in Minkowski space are straight lines, it so happens that in this special case the expression for the distance between the coordinate points $x^\mu$ and $0$ is given by acting as if $x^\mu$ is a vector and computing its norm with the metric tensor given by the $\mathrm{d}s^2$ expression. Doing so directly, however, is formally wrong because you cannot apply a pseudo-Riemannian metric directly to points in that fashion. So in this case the question is doubly wrong: It does matter, in principle, where the indices are placed and you cannot even write something like $x_\mu$ for a set of coordinates.

Group indices

The usage of indices in group theory is completely different, and a priori there is no notion of "upper" or "lower" indices. Given a group $G$ and a representation $\rho : G \to \mathrm{GL}(V)$ of some vector space $V$, one can of course choose a basis $v_i$ of $V$ and write any group element as a matrix $\rho(g)_{ij}$.

The notion of upper and lower indices enters here for groups where all or most of the irreducible representations can be constructed from tensor products of the fundamental representation: One declares vectors in the fundamental representation to have components with indices $v_i$ and those in the conjugate fundamental representation to have components with $v^i$ (or vice versa) and then one can write $T^{\mu_1\dots\mu_m}_{\nu_1\dots\nu_n}$ to denote an element of $\bar{V}^{\otimes m}\otimes V^{\otimes n}$. This shorthand is useful to then deduce which combination of indices and their (anti-)symmetrization correspond to irreducible representations, see e.g. this answer.

Again, the upper and lower indices are related, but do not denote the same objects, and they signal a different transformation behaviour under the group (fundamental vectors transform by $\rho(g)$ while anti-fundamental vectors transform by $\bar{\rho(g)}$), just like indices in the geometric case signal different transformation behaviour under coordinate changes.