This might be a very trivial question, but in condensed matter or many body physics, often one is dealing with some Hamiltonian and main goal is to find, or describe the physics of, the ground state of this Hamiltonian.
Why is everybody so interested in the ground state?
To add to Roger's answer, the ground state is interesting because it tells us what the system will do at low temperature, where the quantum effects are usually strongest (which is why you're bothering with QM in the first place). OR it is interesting because the finite temperature behavior can be treated as a perturbation above the ground state.
For example, in a metal, the dividing line between "low" and "high" temperature might be the Fermi temperature (essentially the temperature that is equivalent to the highest occupied electron state). For many metals the Fermi temperature is on the order of $10^4 K$ or more, so a metal at room temperature is nearly in its ground state, with a few excitations given by Fermi-Dirac statistics.
As another example, if you consider a permanent magnet, the relevant temperature scale is the Curie temperature which might be hundreds of K, so a room temperature magnet could be considered to be in its ground state with some excitations (perturbations) on top of that.
Ground state contains information about most thermodynamic properties of the system at zero temperature. In fact, it can be thought of as a limiting case of the partition function at zero temperature. In many respects many physical systems never depart far from their ground state (although this notably not the case when dealing with phase transitions).
Obviously, there are many problems - notably all kinds of dynamical problems, such as relaxation or transport phenomena - that cannot be reduced to studying ground state.
It is also worth pointing out the ambiguity of the language: we are not literally interested in the state or the absolute value of its energy, but rather how this state and its energy come about from various types of interactions, and how they depend on the parameters.
Update
Here is my answer to another question that underscores the special role of the ground state in optics.
The equilibrium properties at low-enough temperatures (for metals at room temperatures, $k_BT\ll E_F$ where $E_F$ is the Fermi energy) can be determined by knowing the properties of the ground state.
At any temperature, $\mathrm{T}$, the equilibrium state of a system is dictated by the minimization of its Helmholtz free energy: $$F=U-T S\tag{1}$$ where $$U=\sum_{n} p_{n} E_{n} \quad{\rm where}\quad p_{n}=\frac{\exp \left(-\beta E_{n}\right)}{\sum_{n} \exp \left(-\beta E_{n}\right)}.\tag{2}$$ At sufficiently low temperatures, it is the minimization of $U$ that essentially determines the equilibrium state. Moreover, at low $\mathrm{T}, U$ can be well-approximated by the ground state energy $E_{0}$: $$U \approx E_{0}\tag{3}.$$ Equilibrium configuration can be determined by knowing these states.
I'm going to give a few examples to add to the other answers.
Spin liquids are low-temperature magnetic phases of matter which do not spontaneously break any symmetries. Generally, some type of frustration prevents the system from adopting any particular ground state, the origin of frustration could be competing energetic interactions or can be caused by geometric arrangements of the magnetic ions which prevent any ground state from being selected, and thus the system avoids ordering, remaining "liquid like". There are both classical and quantum versions of spin liquids. Classically, they are characterized by a macroscopically large set of ground states, such as the Kagome and pyrochlore antiferromagnets, and correspondingly a non-zero entropy at very low temperatures. These systems are interesting because they exhibit interesting emergent properties: in particular, the dipolar pyrochlore magnets Dy$_2$Ti$_2$O$_7$ and Ho$_2$Ti$_2$O$_7$ exhibit emergent magnetic monopole excitations. In the quantum case, one can have a massive superposition of the degenerate classical groundstates, similar to Anderson's Resonating Valence Bond (RVB) liquid model, originally proposed to explain some of the properties of high-temperature cuprate superconductors, and these superpositions generally lead to a large amount of entanglement, meaning that the ground state is not a product state. These quantum spin liquids can have all sorts of interesting properties, and on the pyrochlore lattice can exhibit emergent Quantum Electrodynamics (QED), including an emergent photon excitation. The pyrochlore spin liquid (called quantum spin ice for reasons I won't explain here) is an example of a gapless spin liquid: the photon excitation is gapless, meaning it requires only an infinitesimal amount of energy to excite the system. Much more common are gapped spin liquids, which are easier to understand: since they are gapped, at low temperatures the ground state will be stable and excitations will be exponentially suppressed. It is then possible (using the methods originally developed to my knowledge by Xiao-Gang Wen) to integrate out the excitations and obtain a gauge theoretic model of the low-energy spin liquid phase, which can include many interesting topological properties. A famous example is the Kitaev spin liquid, which has anyonic excitations.
The key to spin liquid physics is that the ground state is highly entangled and does not break any symmetry, in contrast to systems like ferromagnets whose ground states are symmetry broken states. Understanding the nature of the ground state wavefunction allows one to understand the low-lying excitation spectrum and describe the low-temperature physics.
One can also study quantum phase transitions: zero-temperature phase transitions which occur as an external variable such as magnetic field or pressure are varied. These are entirely described in terms of the change in the ground state of the system at some critical value of the external control parameter, a simple example being the transverse field Ising model.
Symmetry broken groundstates are also of interest and are plentiful and easy to find. The simplest example probably is the ferromagnet, which has a global spin rotation symmetry which is spontaneously broken at the critical temperature as the system orders. The ground state of a ferromagnet is a simple product state (all spins point in the same direction, i.e. an effectively classical state), which by itself is unremarkable, but one can still understand the low energy excitations (magnons) once one knows the ground state. Some more complicated quantum examples of symmetry broken phases are superfluids, (BCS) superconductors, and Bose-Einstein Condensates (BEC's). Both superfluids and BCS superconductors can be thought of as BEC's in a way, in that they are "adiabatically connected to" a BEC state, i.e. I can "continuously deform" the ground state wavefunction to reach a BEC wavefunction. Again, for the purposes of understanding the low-energy physics, understanding the ground state is crucial, as it contains much of the interesting useful information for understand the low-energy properties of these systems.
Apart from the answers already provided, one reason why ground states are studied is because they are quite robust. Leave a system, coupled to a bath (a system with much larger degrees of freedom), then the state of the system tends to be the one that minimises the overall energy, ie, the ground state.
This is because the ground state cannot be de-excited further and that makes it comparatively long lasting. The excited states have multiple ways in with they can relax to a lower state. And the probability of that transition is proportional to the number of ways it can decay, according to the fermi golden rule.
This indirectly tells you that equilibrium properties are heavily dependent on the ground state properties. And most situations can be treated perturbatively.
