I get the idea of traveling to the future and it makes perfect sense as we'd be somehow trapped in a machine that will travel at a very high speed (close to c) thus slowing down our aging wheres time would flow normally in, let's say, Earth. So years would pass faster for people on earth than for the person in the time machine, making it virtually possible to travel to the future. However, I don't see how traveling back in time could be possible following this logic.
First of all, you can easily travel in space to some place and go back in that place anytime you like. But you cannot travel at a date and go back to that date anytime you like.
So whatever your speed is close to c or not you have a spacetime trajectory that allow you to move in space and time but we are like in a boat in the rapids moving at a certain speed and you cannot trace back the rapid but just merely manage your direction to avoid obstacles. If someone was following you from the bank of the river then he would always go forward sometime slowing down and sometime accelerating.
Therefore, travelling back would be illogic following that logic!