I assume that you are asking about the time for the same mass of water to reach the same equilibrium temperature from the same number of degrees above or below.
Heat transport is a diffusion process. The rate of diffusion depends on the speed of the particles which transport thermal energy. If heat is being transported by conduction or convection, then the higher the molecular speed is the faster will be the rate at which heat is transported. Molecular speed increases with temperature, so yes, the water which starts at the higher temperature will reach thermal equilibrium quicker.
However, the difference in times is usually very small for typical differences in temperature. The same mass of water at 25C will cool to 20C only very slighty quicker than it would warm up to 20C from 15C. These differences should be seen on the Kelvin (thermodynamic) temperature scale, on which one is a decrease of 1.678% and the other is an increase of 1.736%. They are further reduced by the fact that the rate of diffusion is proportional to molecular speed, which is proportional to the square root of thermodynamic temperature, so the experiment is actually comparing speeds of 1.302 and 1.318 (using some arbitrary unit). The % difference in the times to reach equilibrium is only about 1.2%.