Limits of zooming in (for camera/telescope) What limits the ability of a camera/telescope to zoom in on distant objects? 
The question is twofold:
What zoom really is?
Firstly, I would like to gain clear understanding of what we mean by zoom and zooming in. It seems that in the case of a camera/telescope we mean something different from the case of a magnifying glass (I have encountered words like magnification, angular resolution, virtual/real image, but I do not have full understanding.)
What limits the zoom?
Diffraction limit is a well-known answer, which is however relevant only to the highest-quality telescopes in space, such as the Hubble telescope. When it comes to our everyday life, there are many other candidates: resolution of the optical sensor, aberrations, lens imperfections, atmospheric transparency, etc. I would appreciate ranking these in importance and, if possible, basic quantification. 
Remark: magnification vs. angular resolution
My understanding is that magnification refers to the case where the object is placed at a distance less than the focal distance behind the lens. In this case the lens creates a virtual image of the object, which has dimensions bigger than the actual object. The magnification then equals
$$ M = - \frac{S_2}{S_1} = \frac{f}{f- S_1},$$
where $f$ is the focal distance, $S_1$ is the distance between the object and the lens, and $S_2$ is the distance between the lens and the observer$.
This is not the case when we deal with a camera/telescope: in this case a real image of an object is created, which is projected on the optical sensor. This real image is actually smaller(!) than the object (which is manifestly obvious in case of observing stars). The "magnifying" effect in this case is actually the increase in the angular resolution which makes the object appear closer rather than bigger.
 A: "Zooming in" means changing the magnification. In other words, zooming in increases the apparent size of objects in the image.
If all you have is a single lens of fixed focal length (i.e. a magnifying glass), the only way you can change the magnification is by physically moving the lens relative to the object. As the distance between the lens and the object approaches the focal length of the lens, the image gets larger and larger.
Of course, in many situations, getting the right distance away from an object to achieve the desired magnification is impractical or impossible. For those situations, we turn to systems of multiple lenses, like telescopes, microscopes, or cameras. In these systems, the magnification is controlled by adjusting the focal length (in telescopes and microscopes, switching out the eyepiece; in cameras, switching out the front lens) and/or the separation (in cameras, moving the components of a parfocal zoom lens) of the various component lenses.
It is impossible to rank the various factors affecting the clarity of a zoomed image without knowing the specific design of the optical system in question. They could be in almost any order of importance depending on the details of the system's design.
A: Although @probably_someone already gave a really nice answer I will try to further clear the confusion.
Zoom
Lets start with digital zoom. You have a picture in your computer, for example 6000x4000 pixels. It is fixed, you cannot change it, was taken in the past. You can display it in your monitor to occupy an area of 3x2 cm or you can display it to occupy 6x4 cm. In this case you did a 2x digital zoom but the area of the picture after is 4x bigger than the original one. A tree in the image will as well occupy 4x more area, but the height will be 2x. The field of view (the angle that comprises the full diagonal, lets imagine 10°) and the minimally resolvable angle were always the same. The smallest angle being encoded in the pixels, for example a branch of a tree 20m away. A tree 40m away does not have its branches resolved.
Optical zoom. You have a zoom lens. For simplicity lets assume its focal lengths are just 50mm and 100mm. The imaging field (sensor/film/white paper and you trace the image) for pictures taken with this lens is the same, but when you use the 50mm, the field of view is larger than with the 100mm. In this case, images taken with this camera/lens combo of the same subject and same distance will yield images 4x bigger in area for the 100mm when compared to a photo taken on the 50mm end (the diagonal or FOV is 2x bigger). The imaging surface is the same in both cases, just the whole imaging apparatus in front of the surface changed. This changes of course field of view and in principle the minimal angle of resolution. But what is important, as zoom, is the relative area of the image once again!
When it comes to a telescope and our eyes, imagine the back of your eye being the imaging surface, similar to the sensor in a camera. The telescope is the same, but different eyepieces will change the total magnification. It is like going from the 50mm case to the 100mm case. The whole assembly, from your eye lens, to the eyepiece to the telescope will create an effective magnification. Now there are lots of things to consider. For example, just magnifying the image of a telescope (meaning a portion of the night sky occupying a larger area on your retina) means that also the total light gathered by the telescope will now have to be spread on a larger surface and the image will be dimmer. In the case of a telescope you cannot also go beyond its minimal resolvable angle (probably_someone already mentioned that) as the telescope is the heavy horse in the imaging, the main "photon gatherer". If you have good eyepieces, high transmission of light and low aberrations, then you can also have a good image. If you completely scratch the surface of one of the objective lenses you are not going to see too much, or in focus.
Please notice that a camera lens and a telescope are very different in construction and how they end up imaging things, but taking the example of the light occupying a larger area, that is exactly the same in the case of camera objectives. If for both the 50 and 100mm lenses the aperture (in simple terms, diameter of the front element) is the same, the 100mm picture will be 4x dimmer!
Zoom is just the ratio of some height between 2 images, after manipulation.
Magnification
The moon, as seen from earth, is a disk occupying approximately 0.5° in diameter of the night sky. At the entrance of your imaging apparatus, this is its occupied angle. At the exit of your imaging apparatus, the angle is (tracing where the edges of the moon are going, from lens to imaging plane), for example 1.5°. You have a magnification of 3x (for small angle approximation). This means that at your location you can now perceive the moon as having a larger angular size. This also means that the virtual image of the moon, at the distance from earth to the moon, is like the moon is 3x bigger in height (as per your example). But remember that you need to be imaging the moon! It does not work until you create a real image and the angles pertain to this image (in case you think of just changing the distance, but keeping the same angle, but then you do not have a real image anymore!). A different case: with macro lenses you many times have 1:1 imaging at closest possible focus. This guarantees that the size (area) of the created image on the sensor, is the same area of the object. A 1cent coin would look the same on a piece of paper on the focusing plane of such a lens.
Angular resolution
This is what you want, in my opinion. This is nothing but the smallest angle which you can resolve. This is I think easier to picture using stars. The smallest angular resolution is that one where 2 close stars in the sky can still be seen as 2 stars, and not a blobby mess. That means which is the smallest distance in angle that the 2 stars would still be resolved as 2, and not one. In the case of a telescope, the telescope defines this angle and then your eyepieces or imaging objectives do the rest of the job for the final magnification/total area/field of view occupied by the picture. With the example above, magnification gave the apparent moon size, but says nothing about the minimal angular resolution. I could use 2 different telescopes, and different eyepieces to obtain the same magnification, but ultimately, how sharp the craters are and the smallest ones I can resolve depend on each telescope and its subsequent optics. If I digitally zoom in on a blurry picture, I will not increase my minimal angular resolution.
To wrap up, the minimal angular resolution of a whole apparatus boils itself down to many many parameters. It is hard to pin-point the bottleneck. In the case of astro imaging, if you do narrowband imaging, then you can always focus your telescope "better" because you don't have to deal with chromatic aberrations.
If you look for commercial, private use telescopes, most of them always say what is the minimum arcsecond resolution, typically diffraction limited. You will see it always depends on the diameter of the aperture. Typically, the focal length of the telescope determines just the maximal field of view obtainable (basically where your imaging circle falls by $1/e^2$ in light intensity). Now you need to play with both numbers. Do you want your field of view to just be a few times of your minimum resolution (meaning everything is a blurry and very very dim mess), or do you compromise and make your field of view larger, so that you can have many many times the minimum angular resolution and the image also brighter?
