Why do we consider heat transfer from/to reservoirs a reversible process?

In order to find the production of entropy on a process between a system and a reservoir, it's useful to consider that reservoir passes through a reversible process. Why is that?

The definition I have for reversible process is that it is a process that can be reversed without leaving marks on the surroundings.

So, if I transfer heat from my system to the reservoir, its temperature wouldn't change. Also, if the reverse happened it wouldn't change the temperature either.. so it could be approximated to a reversible process.

You seem to have misunderstood what a reversible process is. In a reversible process entropy of the universe doesn't change. Reversible process does not refer to particular systems within the universe. Now if the temperature of the reservoir and system are different, then heat transfer between them is irreversible, resulting in entropy change of the universe equal to $\lvert Q/T_{reservoir}-Q/T_{system} \rvert$. Just the fact that the temperature of the reservoir remains constant does not make the process reversible.