This question really belongs on chemistry.SE, which is still in area 51.
While answering this question, I realised that there was a flaw in the standard logic for these situations.
Let's take $NaNO_3$ solution in water, and compare it with pure water (same size cell). (I'm not taking $NaCl$ for a reason*).
In both cases, we have nearly the same(~7) PH and pOH, right? So concentrations of $H^+$ and $OH^-$ are the same, and thus contribution of these ions to the overall conductance/conductivity is the same. These ions both migrate to the electrodes, get reduces/oxidised, and emit/absorb electrons, facilitating passage of current.
OK. Now, let's consider the $Na^+$ and $NO_3^-$ ions. Yes, they migrate as well. But, they don't get redoxed (the water ions are preferentially redoxed). So, all I see happening here is a buildup of charges on either electrode, which will stop once equilibrium is attained. This buildup of charges cannot migrate to the outside circuit like a capacitor, as it cannot translate to electrons. But, these ions still make a significant contribution to the net conductance by Kohlrausch law.
So how does a general good neutral electrolyte help water conduct electricity? I feel that it should conduct electricity at the same rate as water; but by Kohlrausch law, it clearly doesn't. And anyways, I've always heard that impure water conducts electricity better.
*$NaCl$ has the issue of overvoltage of $O_2$, causing oxidation to of $Cl_2$, which complicates the situation. The $Cl^-$, being able to get oxidised, facilitate current and thus don't serve as a good example here.