In a neutral conductor if we assume electrons as point charges, the electric field in the space between them cannot be identically zero. This microscopic field may be very weak. What if we were very close to one of electrons? Shouldn't the electric field diverge?
How should it be “very weak”? Its field strength is immense, against macroscopic samples. Sure, one shouldn’t suppose a Maxwellian E-M fields inside a matter, especially an electric conductor – this microscopic field is uncertain. One hardly can understand it thinking about it as a vector or tensor field in the spacetime; one needs QFT. There is a related discussion at Ambiguity on the notion of potential in electrical circuits?.