To begin, note that there is more than one incarnation of "the" harmonic oscillator in physics, so before investigating its significance, it's probably beneficial to clarify what it is.
What is the harmonic oscillator?
There are at least two fundamental incarnations of "the" harmonic oscillator in physics: the classical harmonic oscillator and the quantum harmonic oscillator. Each of these is a mathematical thing that can be used to model part or all of certain physical systems in either an exact or approximate sense depending on the context.
The classical version is encapsulated in the following ordinary differential equation (ODE) for an unknown real-valued function $f$ of a real variable:
\begin{align}
f'' = -\omega^2 f
\end{align}
where primes here denote derivatives, and $\omega$ is a real number. The quantum version is encapsulated by the following commutation relation between an operator $a$ on a Hilbert space and its adjoint $a^\dagger$:
\begin{align}
[a, a^\dagger] = I.
\end{align}
It may not be obvious that these have anything to do with one another at this point, but they do, and instead of spoiling your fun, I invite you to investigate further if you are unfamiliar with the quantum harmonic oscillator. Often, as mentioned in the comments, $a$ and $a^\dagger$ are called ladder operators for reasons which we don't address here.
Every incarnation of harmonic oscillation that I can think of in physics boils down to understanding how one of these two mathematical things is relevant to a particular physical system, whether in an exact or approximate sense.
Why are these mathematical models important?
In short, the significance of both the classical and quantum harmonic oscillator comes from their ubiquity -- they are absolutely everywhere in physics. We could spend an enormous amount of time trying to understand why this is so, but I think it's more productive to just see the pervasiveness of these models with some examples. I'd like to remark that although it's certainly true that the harmonic oscillator is a simple an elegant model, I think that answering your question by saying that it's important because of this fact is kind of begging the question. Simplicity is not a sufficient condition for usefulness, but in this case, we're fortunate that the universe seems to really "like" this system.
Where do we find the classical harmonic oscillator?
(this is by no means an exhaustive list, and suggestions for additions are more than welcome!)
- Mass on a Hooke's Law spring (the classic!). In this case, the classical harmonic oscillator equation describes the exact equation of motion of the system.
- Many (but not all) classical situations in which a particle is moving near a local minimum of a potential (as rob writes in his answer). In these cases, the classical harmonic oscillator equation describes the approximate dynamics of the system provided its motion doesn't appreciably deviate from the local minimum of the potential.
- Classical systems of coupled oscillators. In this case, if the couplings are linear (like when a bunch of masses are connected by Hooke's Law springs) one can use linear algebra magic (eigenvalues and eigenvectors) to determine normal modes of the system, each of which acts like a single classical harmonic oscillator. These normal modes can then be used to solve the general dynamics of the system. If the couplings are non-linear, then the harmonic oscillator becomes an approximation for small deviations from equilibrium.
- Fourier analysis and PDEs. Recall that Fourier Series, which represent either periodic functions on the entire real line, or functions on a finite interval, and Fourier transforms are constructed using sines and cosines, and the set $\{\sin, \cos\}$ forms a basis for the solution space of the classical harmonic oscillator equation. In this sense, any time you are using Fourier analysis for signal processing or to solve a PDE, you are just using the classical harmonic oscillator on massively powerful steroids.
- Classical electrodynamics. This actually falls under the last point since electromagnetic waves come from solving Maxwell's equations which in certain cases yields the wave equation which can be solved using Fourier analysis.
Where do we find the quantum harmonic oscillator?
- Take any of the physical systems above, consider a quantum mechanical version of that system, and the resulting system will be governed by the quantum harmonic oscillator. For example, imagine a small system in which a particle is trapped in a quadratic potential. If the system is sufficiently small, then quantum effects will dominate, and the quantum harmonic oscillator will be needed to accurately describe its dynamics.
- Lattice vibrations and phonons. (An example of what I assert in point 1 when applied to large systems of coupled oscillators.
- Quantum fields. This is perhaps the most fundamental and important item on either of these two lists. It turns out that the most fundamental physical model we currently have, namely the Standard Model of particle physics, is ultimately based on quantizing classical fields (like electromagnetic fields) and realizing that particles basically just emerge from excitations of these fields, and these excitations are mathematically modeled as an infinite system of coupled, quantum harmonic oscillators.