Many of the other answers are addressing the practicalities of expanding in Fourier series versus Taylor series. But there is at least one *physical* reason for choosing one over the other, and that is that the expansion coefficients of a vector written in an orthonormal basis reveal particular types of *physical* information about the system being described by the function, and the type of physical information that is revealed depends on the choice of basis:

**The physical relevance of expansions in an orthonormal basis**

This is directly related to some of the answers here that cast Fourier series in the language of linear algebra, where the sines and cosines compose an orthonormal basis for the space of functions you are looking at. In this context, an orthonormal basis has four nice properties.  The first is merely that orthonormal bases are nice calculationally in that they allow you to easily compute expansion coefficients by taking the inner product of the vector with a basis element. The other three are more physically relevant:

- The expansion coefficients in an orthonormal basis have a direct physical interpretation. In the context of basic mechanics, for instance, the $v_x$ attached to $\hat{x}$ in a velocity vector *really is* the velocity in the $x$ direction, in the following sense. Imagine moving along the $x$-axis at velocity $v_x$ (of the object) and watching the object as it moves: the object will recede from you but keep pace with you along that direction. So, importantly, **the components of a vector in an orthonormal basis reveal physically relevant information about the system described by that vector**.

- The type of physical information that's revealed depends on the orthonormal basis that's chosen. If you choose to represent a velocity vector in the basis $\{\hat{x},\hat{y},\hat{z}\}$, then you get the velocities in these directions (in the sense explained in the previous bullet point). If instead you choose the basis where one of the basis vectors lies along $\vec{v}$, then the component along this basis vector is the *speed* of the particle. Different bases reveal different information and conceal different information.

- Finally, the expansion coefficients in an orthonormal basis are *unique*, which is what really allows us to interpret the quantities physically. (What would the $x$-component of velocity really mean if it could take on two different values?)

**The physical interpretation of expansion coefficients in a Fourier series**

In the context of Fourier series, these bullet points have the following meaning. 

- The expansion coefficients tell us how much the function looks like a particular sine or cosine function. Let's consider one of the terms in the expansion, given by
$$a_n\cos\left(\omega_n t\right) + b_n\sin\left(\omega_n t\right),$$
where $\omega_n=2\pi n/T$. If we re-write this as
$$A_n\sin\left(\omega_n t+\phi_n\right),$$
where $A_n=\sqrt{a_n^2+b_n^2}$ and $\phi_n=\tan^{-1}(a_n/b_n)$, then we can interpret the coefficients as giving us two pieces of information. The amplitude $A_n$ tells us how much this particular function comes into the expansion, and therefore how much the original function looks like this particular sine function. In other words, $A_n$ tells us to what extent the original function oscillates with frequency $n/T$. In other words, the expansion coefficients are giving us the *frequency content* of the original signal, where the expansion coefficients quantify this relationship. (The phases $\phi_n$ would be relevant in, for instance, interference experiments, but for the purposes of this discussion, it's the amplitude that really matters.)

- A Fourier series (and a Fourier transform) yields the frequency (or wavelength, depending on the context) content of the function that is describing some physical quantity in some physical system. One can use other sets of orthogonal functions, too. For instance, in electrostatics, we can expand the electrostatic potential using Legendre polynomials, and these functions yield the$^1$ multipole (monopole, dipole, quadrupole, etc.) structure of the electrostatic potential. Different orthonormal bases reveal different types of *physical* information about the system.

- Uniqueness again allows us to do the above. If the Fourier series was actually *a* Fourier series$^2$, then this wouldn't really work.

**Taylor series**

As far as I understand, there is no "natural" inner product on the set of polynomials that makes the set of monomials $x^n$ orthonormal, and so we can't really interpret a Taylor series as an expansion in an orthonormal basis. However, we *can*$^3$ interpret a Taylor series as an expansion in a non-orthogonal basis, and so we get back a little of the interpretational power.

In particular, the expansion coefficients reveal *structural* information about the function, in that it reveals how *constant* the function is, how *linear* the function is, how *quadratic*, how *cubic*, and so on. This can have important physical consequences as well, as evidenced by the fact that the simple harmonic oscillator is ubiquitous: if a potential energy is "very quadratic", then the behavior of the system is very much like a simple harmonic oscillator.

This is *not* the type of information that is revealed via a Fourier series$^4$, and hence **different bases reveal different information**.

**Numerical approximation**

Finally, as mentioned in other answers, one or the other can aid numerical computations, but I would argue that this *also* has physical implications (or is at least informed by the physics). As an archetypical example, just see the previous section about Taylor series. In some sense, the approximation of the system as being a simple harmonic oscillator is a *numerical* approximation, and it is one that has both physical meaning and physical justification.

---

<sub>1. All but the first non-zero term in the expansion actually depend on the origin chosen, so using "the" here is not correct, and there's a little sophistry here in the sense that I'm stressing the importance of the uniqueness of the expansion. The point is still a good one, though.</sub>

<sub>2. We can of course choose different intervals over which to compute a Fourier series for a particular function, and in that sense a Fourier series is not unique. However, the interval is usually determined by the physical system we are describing, and so in that sense we get uniqueness back. In fact, sometimes we are actually computing a Fourier *transform*, in which case the interval is the entire real line.</sub>

<sub>3. In physics we can, anyway, to get an intuitive feel for what we're doing. And *perhaps* we can in mathematics, too, to the extent that they allow us. I don't know to what extent mathematicians think of Taylor series in this way, though.</sub>

<sub>4. Except that, as mentioned in another answer, there is a relationship between Fourier series and Taylor series that is revealed in the context of complex analysis. So there's another bit of sophistry in the name of getting my point across.</sub>