Questions first, then my rough estimations:
Is it possible to perform moon laser ranging with amateur motorized 114 mm telescope? My calculations suggest that for 1 mJ laser it should receive ~2 photons per source 1 mJ laser pulse.
Given that we already talking about individual photons, how it was possible to perform moon laser ranging BEFORE retro-reflectors were deployed to the moon? Retroreflector sends back light in ~1 arcsecond angle, while bare lunar surface - in ~6 archours, which means we supposed to receive signal $\sim(6\times 60 \times 60)^2 = 4.5 \times 10^8$ weaker, i.e. even with 2.5 meter telescopes we are talking 1 photon per 250 pulses of 1 J each.
My rough estimations: Given that atmosphere turbulence limits telescope resolution to ~1 arcsecond (adaptive optics was not available when laser ranging experiments started, nor it is available now for amateurs), if we use telescope with diameter larger than ~150 mm (so that we are limited by atmosphere, not diffraction) to expand the laser beam we will get ~1939×1939 meter illuminated area on the moon surface ($\tan(1\:\mathrm{arcsec}) \times 400,000\:\mathrm{km}$). Which means only 1/(1939×1939) part of our energy will reach reflector.
Retroreflector is ~1×1 meter in size. It will reflect the light with same beam divergence - 1 arcsecond. Too sad, as diffraction limit for retro-reflector of such size is ~0.2 arcsecond.
So, if our receiving telescope has area of ~1 $\rm m^2$, we will receive again 1/(1939*1939) part of what reached the moon, so total attenuation is ~$1.4\times 10^{13}$.
If we use 532 nm pulse laser with 1 mJ pulse energy, it will emit 2.67*10^15 photons, which means we are going to receive ~190 photons per pulse. Sounds realistic.
These calculations suggests that 114 mm amateur telescope should be able to detect 2 photon per pulse - again should be detectable statistically.