Time travel is difficult to prove or reject, even by using a formula (at this time there is no decisive proof for or against it). There are solutions of Einstein's equations which allow closed timelike curves, like Gödel's universe, Kerr black holes, wormholes, etc.
As mentioned in other answers, chronology protection conjecture, grandfather paradox and other causality paradoxes, and the absence of visitors from the future are the most used arguments against time travel. Each of them can be avoided, by creating alternative universes in which you don't exist because you killed your grandfather, or by protecting the past using Novikov's self-consistency principle, or a time patrol or similar solutions, etc.
Also, there are arguments against particular time travel methods. For instance, that you can't use wormholes, because they will become singular or/and self-destroy. There are theorems suggesting that in reasonable conditions closed timelike curves are accompanied by singularities, so if there are no singularities, there is no time travel.
Alternatively, you can make the characters discover fictional laws forbidding time travel.
I am very curious why a science fiction writer would want to reject time travel.
Update
If you are interested in various aspects of time travel (involving formulae), check this conference, The Time Machine Factory, 2012. The proceedings just appeared and are available online.
Update 2
The introduction of this paper by Seth Lloyd et. al. contains a brief review of many results concerning time travel, and references.