No, not for any amount that could reasonably be called "slight".
The thinking error I see here is that I believe what you are imagining happening is the Moon to start "spiraling out of orbit", like a plane going down without its engines, if it is disturbed somehow. And that this is something one might often pick up from movies or other such casual, pop-culture media or references thereto, or simply again by very Earth-bound based intuition of how things work such as that moving vehicles like planes need constant control or they will crash, but that is not actually correct in astrodynamical situations.
The more correct intuition is this. In astrodynamics, no matter what sort of "pull" you feel when thinking about this, a small body orbiting a larger one in vacuum, under the sole influence of gravitational forces alone, assuming at least that larger one is very close to spherical or one is far enough away that it is "pointlike" enough, will never fall out of orbit by itself. "Nudging" it won't work either, because orbits are stable in the sense that there is an extremely wide range of parameters of speed and distance over which the two bodies will just keep on orbiting. Hence, the "nudge" will simply shift the shape of the orbit slightly.
Generally speaking, a nudge of the orbiter "from behind" will cause it to travel a bit further out, because it's now going a bit faster and hence can more "successfully" fight the gravity of its parent body before being pulled back. Conversely, a nudge the other way, slowing it down, will cause the orbit to shrink somewhat and it will fall closer to the parent body for the opposite reason. A nudge from an oblique angle will have an effect somewhere in between.
Also, note that when I say "shrink", I don't necessarily mean the orbit simply uniformly gets smaller. (Closed) orbits are ellipses, and such nudging will actually tend to stretch or squeeze the ellipse.
Of course, one might, then ask, what this "orbital decay" one may have heard of is. Well, orbital decay is what happens when you add some kind of friction or drag process into the system that causes it to steadily lose energy. In that case, the orbit will shrink, so in effect the orbiter "spirals" inward, and it will eventually crash into its parent body. In effect, what is happening is the object is being continually "nudged" by the drag in the direction exactly opposite its orbital motion, until it runs out of speed and finally crashes.
However, drag requires some kind of medium against which to rub, and the vacuum of space is just that: vacuum. This phenomenon is of most concern for something like the International Space Station, which is close enough to Earth that there is actually still some very tenuous atmosphere present around it and for which the slight drag it provides adds up over time. If there were enough friction in space to bring down something as remote, big, and heavy as the Moon (by comparison, at a distance of about 377 Mm from Earth's surface versus the ISS at 0.4 Mm, and with a mass of roughly $7 \times 10^{19}$ Mg, against 420 Mg for the station), it would have been brought down already long ago - and likely our Universe would have to be quite different. Moreover, even with friction present, "nudging" won't suddenly cause some sort of dramatic increase in the orbital decay: again, this goes back to what I said before about "stability".
