Physicists use three vector spaces: Euclidean space, Minkowski space, and a separable Hilbert space (with some exceptions). All of these have a quadratic form (in the first and third, it is an inner product, but it is not positive definite for Minkowski space) allowing one to define angles. (Okay, for the Hilbert space, it is twisted by the complex conjugation, but you get the idea.)
But what does "invariant" mean? Really, it doesn't mean anything because given the quadratic form, then one defines the group $O$ (or, in the complex case, $U$) as the linear transformations of the space that are invertible and continuous and preserve the quadratic (or Hermitian, as the case may be) form. So "invariant" means nothing, it comes for free. Given any form, one can find a group that preserves it, and then it is invariant under that group.
Much of theoretical physics is linear. Some is not. Only when it is linear can one use nothing but vector spaces. Vector spaces are inherently linear. When the phenomena are non-linear, then linear forms (and a quadratic form is really a kind of bi-linear form) become less useful, and have, at any rate, to be supplemented with other, more difficult, constructs. Like flows and foliations etc.
GR is non-linear, and uses curved manifolds, which are not vector spaces. Even then they are locally like vector spaces, and so linear analysis plays an important part of it, and the vector spaces appear as the tangent spaces to a curved manifold. But that was not the original question.