I think the point is that you have some object, which when you naively try to calculate, happily gives you it's perturbative (series) expansion $1+2+3\dots$. The mathematician then sees the mistake and recognizes the object for what it is, and tells you the answer.

Further reading:
https://www.google.com/search?btnG=1&pws=0&q=when+does+a+function+equal+to+its+taylor+series

Incidently, one of the places you encounter this sum in physics is in bosonic string theory, where each number summed represents the contribution of a mode on the string. Something similar also arises in the [casimir force][1] calculation


  [1]: http://en.wikipedia.org/wiki/Casimir_force