Your intuition is correct and physically important but a little more complicated than you describe
When we say atoms are electrically neutral we mean that objects far away from the atom see no net electrical field (atoms are very small, so even if the effect you envisage exists it will only be apparent for objects close to the atom).
But your simple intuition is complicated because electrons in atoms are not point particles existing at a single point in the atom. They are more like standing waves spatially distributed around the nucleus (because of Heisenberg and quantum mechanics). This means things are a lot more complex than they would be if electrons were simple point particles revolving around the nucleus generating a moving electrical dipole.
But, the basic intuition that the overall field generated by the electron clouds surrounding atoms and molecules is not zero is correct. And this is very important in chemistry and for the physical properties of molecules.
The electrons in asymmetric molecules are often distributed in ways that generate electrical dipoles. The simple hydrogen chloride molecule (despite being neutral overall), for example, has a strong electrical dipole because chlorine pulls electrons away from hydrogen.
But this probably isn't the key thing you were asking about. Even neutral molecules without built-in dipoles have some features where the distribution of electrons can generate momentary electrical fields. While the simple picture of an electron as a point particle revolving around a nucleus isn't a good way to picture this, even quantum mechanics has to admit that there is uncertainty about the electron's position in its orbital cloud. This uncertainty generates temporary fluctuating electrical fields because the distribution doesn't precisely offset the electrical field of the nucleus. The detailed quantum explanation of this is pretty complicated but the picture of fluctuating electrical fields captures some of the idea intuitively.
More importantly these fluctuating fields are very important in the real world. They are the basic explanation of van Der Waals (or, more correctly London) forces between molecules and between atoms. When one molecule has a fluctuating field, it can generate a field in a neighbouring molecule that is close to it. This generates a weak force that declines rapidly with the distance between the molecules (it is proportional to r-6 where r is the intermolecular distance). For neutral molecules with no built-in dipole, this is a major factor in their physical properties. Gases like N2 would never become liquids if the force did not exist. Simple hydrocarbons and many other non-polar molecules would not liquefy at room temperature without these forces. Even in many polar molecules the London force accounts for the majority of the molecule-molecule interaction (see wikipedia).
London forces can even be important at the macroscopic level: they are the underlying force that allows geckos to walk up walls.
So, yes, even neutral atoms or molecules have fluctuating electrical fields because of the mismatch between the "position" of the electrons and the nucleus. And these fluctuations are important because they generate forces between atoms and between molecules. These are very important in determining the physical properties of many substances and can even be seen at macroscopic scales.