Just because a sine wave looks like a side-to-side wave when you plot it, it doesn't mean that anything is actually oscillating sideways.
In general, a wave is just a pattern in some physical quantity that propagates through space. The physical quantity could be something visible, like the transverse displacement of a string or the longitudinal compression of a slinky, or it could be something invisible, like a force or the electromagnetic field. It can be something which has a direction, like the examples in the previous sentence, or it can be something which has no direction, like pressure or density. It depends on the medium and the type of wave.
However, we can mathematically describe waves in a way that is independent of the medium by expressing the physical quantity that is "waving" as a function $Q(x,t)$ of position and time. Regardless of which medium or which physical quantity is involved, the function will satisfy the wave equation,
$$\frac{\partial^2 Q}{\partial x^2} = \frac{1}{v^2}\frac{\partial^2 Q}{\partial t^2}$$
In a sound wave, $Q$ could be pressure or density or longitudinal displacement. In a light wave, $Q$ can be any of the components of the electromagnetic field, or any of the components of the EM potential. And so on.
Now, if you're familiar with Fourier analysis, you'll know that any periodic function can be expressed as a sum of (co)sine waves, or equivalently of complex exponentials
$$f(t) = \sum_{n = 0}^{\infty}F_s(n\omega_0)\sin(n\omega_0 t) + F_c(n\omega_0)\cos(n\omega_0 t) = \sum_{n = -\infty}^{\infty}F(n\omega_0)e^{-in\omega_0 t}$$
for some value of $\omega_0$. And this generalizes to non-periodic functions as the Fourier transform,
$$f(t) = \frac{1}{\sqrt{2\pi}}\int F(\omega)e^{i\omega t}\mathrm{d}\omega$$
This is particularly convenient for analyzing waves because it's very easy to analyze what happens when $Q(x,t)$ takes the form of a sine wave. And once you know how sine waves behave, you can use the Fourier transform to add them up to reconstruct the behavior of any general wave.
There is another reason we like sine waves, namely resonance. As waves (sound, light, etc.) travel through space they are eventually going to run into objects and interact with them. In many cases, these objects are sensitive only to certain frequencies, and they will "pick out" only those particular frequencies from the incoming wave, letting the rest pass through untouched. For example, this is how absorption spectra are produced in stars (with light waves, of course). If you imagine the wave as being constructed by adding together sine waves of different frequencies, in the sense of the Fourier transform, then it's easy to understand how an object can pick out only certain frequencies of the wave, and easy to analyze what the effect of any given physical system is on a wave.
\rho_0between dollar signs renders as $\rho_0$, and\omegarenders as $\omega$. The $k$ you see there is called the "wave vector" (yes, even though Jerry has expressed it in one dimension) and codes both the direction and the wavelength. Oh, and welcome to Physics.SE! – dmckee♦ Nov 29 '11 at 19:56