What's the advantage of dealing with conservative forces? Do the conservative forces give us any advantage in dealing with mechanics? Do they confer us any advantages? 
 A: Well, the description of conservative systems (where only conservative forces are present) is immensely easier than in the case of nonconservative systems. Here are some "advantages" that come to my mind:
1) Conservative systems exhibit energy conservation. That means that if the the total energy of the system at one moment in time is $E_0$, the total energy at any other moment (past or future) will be the same. This greatly simplifies calculations and in general the physical description of the system. In equations $$\frac{d E}{dt}=0$$
Basically the energy conservation is the starting point of the Lagrangian and Hamiltonian description of classical mechanics (see also the answer of CR Drost on this point), as well as quantum mechanics. By the way, notice that quantum mechanics is an intrinsically conservative theory. 
2) Conservative forces can be described in terms of fields (scalar or vector fields). This is a necessary prerequisite for many theories, like electrodynamics, special and general relativity.
3) Conservative systems are thermodynamically dead. With this sentence I mean that the first principle of the thermodynamics reduces to the energy conservation $\Delta E=0$ (see previous equation) and that the second principle reduces as well to $$\Delta S=0$$ that is, the entropy of the system is constant. As a consequence any thermodynamical quantity is constant. The system is not evolving from a thermodynamical point of view.
4) Conservative systems satisfy the Liouville's theorem, that is, the phase space volume occupied by a collection of trajectories evolving in time is constant. 
5) Conservative systems do not show any arrow of time (see point 1 and 2). They are completely symmetric under time-reversal $t\to -t$. That means that you will be not able to distinguish whether a movie is played forward or backward in time.
Concluding, conservative systems are predictable, deterministic, and relatively easy to describe mathematically. One may say they are boring. Conservative systems cannot exhibit complex phenomena such as irreversible processes, like most of the biological and chemical reaction, and life. 
A: Yes, they do.
There exist two basic perspectives for classical mechanics. One we could call the explicit or Newtonian perspective, this involves considering vectorial things called "forces" and "momentum" with the basic result that the net force on a particle is the time-rate-of-change of the momentum of that particle. This is a really great perspective when you're trying to analyze, for example, the stresses that are on a bridge as it simultaneously has cars driving across it and is swaying in the wind.
The second we can call the implicit perspective, and this involves considering scalar things called "energies" and sometimes "actions." These two perspectives are very closely tied together; you can derive the energy perspective from Newton's laws and in fact someday you may learn something called "variational calculus" which allows you to define a "Lagrangian function", usually calculated as the difference between the kinetic and potential energies, and you get to recover Newton's laws from a "principle of least action." Theorists love this stuff because it turns out that all of those complicated 'conservation' laws from the explicit perspective become symmetries of the laws of physics in the implicit perspective: for example Newton's third law expresses conservation of momentum; momentum conservation in the explicit perspective is equivalent to "the laws of physics don't change if we take a little step to the left" in the implicit perspective.
Conservative force fields are a core ingredient to looking at the world from the implicit perspective.
Let me give you an example. Let's suppose that you have two forces: a conservative force and a drag force. Your drag force always opposes the velocity, so it always does negative work on the system. Therefore the system is constantly losing energy, both kinetic and potential. We can therefore make a quick prediction: "if I leave this thing alone for long enough, it will be at rest (0 kinetic energy) at a local minimum of the potential energy."
You can use this "minimum energy equilibrium" principle to, for example, rediscover the laws of buoyancy for yourself.
