Why do reversible processes not increase the entropy of the universe infinitesimally? The book Commonly Asked Questions in Thermodynamics states:

When we refer to the passage of the system through a sequence of internal equilibrium states without the establishment of equilibrium with the surroundings this is referred to as a reversible change. An example that combines the concept of reversible change and reversible process will now be considered.
For this example, we define a system as a liquid and a vapor of a substance in equilibrium contained within a cylinder that on one circular end has a rigid immovable wall and on the other end has a piston exerting a pressure equal to the vapor pressure of the fluid at the system temperature. Energy in the form of heat is now applied to the outer surface of the metallic cylinder and the heat flows through the cylinder (owing to the relatively high thermal conductivity), increasing the liquid temperature. This results in further evaporation of the liquid and an increase in the vapor pressure. Work must be done on the piston at constant temperature to maintain the pressure. This change in the system is termed a reversible change. It can only be called a reversible process if the temperature of the substance surrounding the cylinder is at the same temperature as that of the liquid and vapor within the cylinder. This requirement arises because if the temperatures were not equal the heat flow through the walls would not be reversible, and thus, the whole process would not be reversible.

But if the system and surroundings are in fact at the same temperature then why would this process occur at all?
My understanding is that in fact that they are infinitesimally different in temperature so I guess my question is why infinitesimality gets these processes "off the hook" for being irreversible. In other words, why do these infinitesimal changes not correspond to an infinitesimal increase in the entropy of the universe, rather than none at all?
 A: The entropy of the surroundings does change infinitesimally.  But the surroundings are large and such a change does not change the total entropy of the surroundings in any sensible way.
Indeed, one already uses that fact in putting the system through a series of reversible steps.  As you point out, if the temperature of system and surrounding were in fact identical, no heat would flow.  But they are infinitesimally different and so an infinitesimal amount of heat does flow.
The same applies to the surroundings.  It too is undergoing a reversible change.
A: Reversible means we can run the process reverse way without any "strangeness". Reverse the process means turn all the interactions opposite way. Lets say, in some process, you transferred out of the system 10 kJ of heat (sign of heat transferred out of the system is negative so Q=-10 kJ). Reverse the process is to transfer this 10kJ back to system from surroundings (positive sign this time). 
Now, let say, the system is actually hotter than surrounding, so you transfer out this 10 kJ of heat easily (if the system has large heat capacity we can neglect the fact that it cooled slightly). But going back is a little awkward, system is still hotter and we want to "put heat" back to the system. Its just impossible. But what is potentially possible is to transfer heat back and forth (avoiding any strangeness) without any temperature difference (or infinitesimally small temperature difference - so how small is really infinitesimally small?). 
My conclusion is that reversible processes are potentially attainable but not realistic (something like perfect beauty) - every real process is indeed irreversible. Yet we use it in thermodynamics  as simplification of real processes, and we can obtain relatively simple mathematical equations for them. 
So essentially, the discussion about realism of reversible processes are something like discussion about realism of point concept in geometry. 
A: The question is:

why do these infinitesimal changes not correspond to an infinitesimal increase in the entropy of the universe, rather than none at all?

In general what you say is true: infinitesimal changes in the independent variables cause infinitesimal changes in properties, such as entropy. 
In the special case that the infinitesimal change of the entropy in the system is cancelled by the infinitesimal change in the surroundings, then and only then we say that the process is reversible. 
A: I think part of the confusion comes from the fact that "the surroundings" is not the same as "the universe". The surroundings are just what the system constituted by the piston interacts directly with. The surroundings can themselves be encompassed into other surroundings.
Now, looking just at what is happening between the piston and its surroundings, there are two ways the piston can be brought to a different (let's say higher) temperature by a reversible change (i.e. a change during which the interior of the piston is always at equilibrium - that is, in particular, homogeneous temperature throughout):

*

*The temperature of the surroundings is changed abruptly, then relaxation allows the temperature of the piston to reach the same value as the temperature of the surroundings by a reversible change. In this case, until the piston and the surroundings are at the same temperature, there is a discontinuity between the temperature of the piston and the temperature of the surroundings and the heat flows from the hotter to the cooler body. Clearly, this process is irreversible in the sens that the reverse process, following exactly the same intermediate states, where the heat flows from the cooler to the hotter body, will never be seen.

*The temperature of the surroundings is changed very progressively, so that the difference in temperature with the piston is, to all practical purposes, undetectable; at the end of the process, the piston (and the surroundings) are at the same state as in case 1, however the path followed is very different. Indeed, in this scenario, it is possible to bring the piston and the surroundings back to their initial states through practically the same intermediate states, by changing back the temperature progressively. In this process, all intermediate states are an equilibrium between the surroundings and the piston, which itself is in internal equilibrium (reversible change). Therefore the reverse process is an admissible process, and so it is not only a reversible change for the piston, but also a reversible process for the piston and its surroundings.

The question whether the entropy of the universe increases or not is a different one. In the first scenario, clearly, the non-reversibility of the process will result in an increase of the entropy of the universe. In the second scenario, this will depend whether the progressive change in temperature of the surroundings is done via a reversible or an irreversible process.
A: By definition a reversible process in an isolated system cannot increase entropy. If entropy is increased during a process in an isolated system then the process is irreversible, by definition.
