As far as I understand, there is no "natural" inner product on the set of polynomials that makes the set of monomials $x^n$ orthonormal, and so we can't really interpret a Taylor series as an expansion in an orthonormal basis. However, we can$^3$ interpret a Taylor series as an expansion in a non-orthogonal basis, and so we get back a little of the interpretational power.