Why does an image only form where light rays coming from a single point get reflected or refracted and converge to a common point? I am a high school student and I have read many books and information on the internet about concept of object and image in optics. They all say that where reflected/refracted rays intersect they form an image at the point of intersection.
If I treat extended objects as if they were made of many points and light rays from each point are going in each possible direction (that's why we can take any ray of our choice from a point to for a ray diagram). If that's the case, then there is actually an infinite number of points where light rays coming from different-different points of an object first intersect and then enter into our eyes just like light from an point source does ,but still we don't see those points we only see the object why?
For example, assume light of a single wavelength gets scattered after hitting an object and intersects at a common point (as shown below). It enters into our eyes and is focused at a point on the retina thus it should send the same signal as those rays are coming from a single point because how can anyone know that these are coming from a single point or not? 
 A: I had this same confusion a while back. You want to be careful here. There are TWO different versions of these diagrams describing the same thing but in a different way. Chances are you are looking at one type of diagram but thinking about the other type.
Since the rays drawn on the two diagrams are different rays (coming from the same point vs multiple points) they converge at different locations on the diagram, either on or off where you would place the detector even though the position of the detector never actually changes.

Tracing rays coming from ONE point for ALL possible paths
In the first version, the rays being traced are  all from the same point so in order to see that point in focus all the rays coming from it have to converge at one point on the detector. So in these diagrams the detector is placed where the rays from that one point converge:

https://www.flir.ca/support-center/iis/machine-vision/application-note/selecting-a-lens-for-your-camera/
It has to be this way for things to be in focus because then each point of the detector is only detecting the light from one point on the object. If each point on the detector was detecting the light from multiple points on the object that makes a blurry image.

Tracing rays coming from MULTIPLE points but only ONE path for each
In the second version, the diagram traces the rays from multiple points on the object to show how an image is formed. Since it is tracing the rays from multiple points, to keep things readable it only traces one path per point.

https://www.flir.ca/support-center/iis/machine-vision/application-note/selecting-a-lens-for-your-camera/
In this case, the rays are from different points and so if you placed the detector at where they converge you would end up with a blurry image as described above because now light from multiple points ends up at the same point on the detector.
In this diagram, since the rays are from different points you want them to intersect away from the detector so that they end up at different points on the detector.
Between the two diagrams, the location of the detector doesn't change. What is changing is the location where the rays converge because you are looking at different types of rays in each diagram.

Then there is this last diagram which I find the most confusing:

https://www.chegg.com/homework-help/definitions/focal-length-2
It's one of the other two diagrams, but which is it?
It's really confusing because the rays are parallel which means that if it was coming from a single point, then that point needs to be at distance infinite, however, the rays are spread over the entire lens which means the object has to appear to be finite size it can't at distance infinite (any object of non-infinite size should should appear infinitesmally small at distance infinite.
It could be a very large object and for some reason you are just tracing all the parallel rays coming from different points. But then the fact that the point where the rays converge is labelled as the focal length makes no sense because then it should be blurry. N
Note the difference with both of the the first two diagrams: the point where the rays converge is labelled the focal point whereas the focal point in the other diagrams is not where the rays converge.
The closest I can gather is it is just a made up situation for the purposes of defining the mathematical definition of focal length and not the rays for any real object (except maybe a laser?)
A: You are correct that rays from a object will cross at $C$ and you are quite right that rays which pass through $C$ can be made to converge at a point $D$ on a screen as shown in the diagram.

What is not correct is that an image of point $C$ will be formed at point $D$ on the screen.
What you will get at point $D$ is the overlap of incoherent light rays which on average will cancel each other out and so produce nothing on the screen.
I have "borrowed" a ray diagram from one of the other answers to help explain as to what is going on.

In the diagram light from a point object passes through a lens and forms a point image on the screen just like light form point $C$ passing through a lens and converging at point $D$.
However that is as far as it goes in terms of similarity.
In the borrowed diagram all the rays of light from the object are assumed to be in phase with one another and take the same time to get to the image point ie arrive in phase with one another and form the image.
The equal time of travel for each rays is because rays from object to image which are shorter in length travel through a thicker part of the lens in which the speed of light is the less than that in air.
Note that those rays which are longer travel through a smaller thickness of lens and the lens is so "designed" to make the time of travel the same for all of the rays.
This does not explain why no image is formed at $D$.
For that we must consider the ray which starts at point $B$ and that which starts at point $A$.
They are not necessarily in phase with one another, indeed their relative phase will be changing all the time so that when they arrive at $C$ their relative phase will be changing all the time as will all the other rays that you have drawn emanating from the object.
Those rays may also have different intensities and different wavelength so the source between $A$ and $B$ are incoherent and on average when the waves from those sources are added together produce zero intensity.
Since the light rays at $C$ cancel each other out so must they at $D$.
A: Imagine the pixels on your computer screen. Imagine if the light from a pixel passed through a lens, and each light ray converged on a single point. And each pixel gets its own destination point. Then, you will get a copy of your computer screen on the destination. That's an image.
Now, imagine if the light from a pixel does not converge on a single point, and instead gets spread out by 2x. Yes, you will still see something. But it has been made blurry, as though your pixels have gotten larger, because they have. Each source pixel became a larger destination pixel, and they are now overlapping. Each destination pixel is now made of 1/4 of the original pixel, and 3/4 of the neighboring pixels. This blurring and mixture makes the image quality worse.
Indeed, if you try it out, you will see something out of focus, but you will still see something, when the rays do not converge.
A: You can see because light enters your eyes and hits your retina. You can see clearly when light from each point in the thing you are looking at, hits one particular spot on your retina. When light from several points hits one spot on your retina, or when light from one point hits several spots on your retina, what you see is blurry. After all, if your eye can't tell exactly where some light is coming from, your brain won't know exactly where the thing is.
On a ray-tracing diagram we start with light rays pointing away from an object. An eye looking at them (because some rays go into it) will see the object. However the eye only knows about what is going into it. It will see clearly even if the rays don't actually come from some point, as long as they appear to.
There are three situations. Firstly there is the situation where light from one place meets again at another place, on a screen. In this case every bright spot on the object gives a bright spot on the screen, every red spot on the object causes a red spot on the screen, and so on. We see a picture (image) of the object on the screen.
Secondly, if we take the screen away and look at the light through where the screen was. The light which was meeting at a spot on the screen now goes straight through and into our eye. Our eye though has no way of telling whether the light is coming from the other side or whether it actually starts where the screen is, so we can see the image as if it was still there.
Thirdly, there are situations where the light does not actually meet at a particular point, but it comes into the eye as though it had come from one particular point. Again your eye can't tell the difference, so what you see is exactly the same as if the light had come from that point.
Ray-tracing diagrams can show the theory of what happens, but they are only theory. In practice lots of the rays you draw will go nowhere near your eye, but the places they meet will be the same as for the rays that do go into your eye.
A: 
there is actually an infinite number of points where light rays coming from different-different points of an object first intersect and then enter into our eyes just like light from an point source does ,but still we don't see those points we only see the object why?

That's correct: the light rays from different parts of the object intersect in all sorts of combinations at every point in space, and then spread out again from every point.
However, only rays that intersect in the focal plane will be focused by the lens of your eye to a single point on the retina. And those are all the rays that exist. Each ray intersects many other rays at many different locations in space, but it only hits your retina once. And it passes through the focal plane once. So after you've analyzed the rays that spread from each point in the focal plane, you're done. You would be double-counting the rays if you also considered intersections elsewhere.
Why are the intersections in the focal plane the "correct" ones to count? They aren't, you can pick any set of intersections as long as you count each ray once. But if you don't pick the focal plane, then the rays spreading from each point will end up at multiple points on the retina. If you consider the destination of each ray individually and correctly, you'll find that it ends up at the same point as the corresponding ray in the focal plane did – obviously, since it's the same ray. It's easier to just pick the focal plane and use the convenient fact that each point in that plane maps to a point on the retina.
Re-focusing your eyes changes the location of the focal plane. By doing that, you can see the rays intersecting at whatever spatial locations you want. But most of those intersections are intersections of rays from different parts of the object. Your retina can't determine the direction of a ray that hits a point on the retina, it only measures intensity, so if you focus on a plane where rays from different parts of the object intersect, you'll just see a blur that's a mix of intensities from different parts of the object. To see the object sharply, you have to focus on either the object itself or on a plane where each ray from a point on the object only intersects other rays from the same point. The latter is called an "image" in optics jargon (specifically a real image).
A: Let's study a simple case. Suppose a point-like object whose light passes through a converging lens. The object emits light in every direction, but in my diagram I will draw only 4 rays (in reality there would be rays in any direction).

If you placed a screen where line A is and looked at it, you would see the following projection:

If you take into account every direction, you will see that every point in the screen is lit. That is not an image of our point-like object. However, in the case of B, every ray would converge forming a single point in the screen. That constitutes our real image of the object.

Without getting into detail, for an extensive object each one of its points emits light and can be treated this way. If you place your screen where the rays don't converge, the light from every source point ends up all over the screen. The screen would be illuminated but you would not see an image of the original object. In practice it would probably be a blurred image.
A: First you have to be very clear why you "see" a real object, for example the letters on your screen or your finger.
the finger sends light in all directions from al points. only a tiny fraction enters your ey. If it is adjusted to the distance, all the light coming from one point of your finger gives one point on your retina, different points on different points of the retina. (I leave out what the brain does with the signal the nerves of retina report) If it would nit be one point from one point of the object, the light from different point would overlap , what you see gets blurred. The your finger or some letters 5 to 10 cm from your ey, you can sell see that there are letters or your finger, but you can not see details or read the letters. The lens of your ey is not able to force all light from one point of the object  to converge on one point.
this is the crucial point to understand.
now two kind of pictures  real ones that one can also have on a screen or virtual ones like in a mirror.
from one point of the screen or the location of the screen there is light coming to your ey, all the light coming from a point   gives one point on the retina. So you see the "picture like the real object
for the virtual pictures, the ey does not know where the light is coming from, it makes one point on the retina where the light seems to come from a point
A: I will try to answer this without diagrams.
Light from an illuminated object leaves every point on the surface, spreading outwards in straight lines. For you to see the object, the divergent rays of light need to be brought back together so that they land on your retina in such a way that their relative locations on your retina match the relative locations of the points from which they left the object.
If the object is a pink circle, say, then the diverging light rays from each point on the circumference of the circle need to be brought back together to land in a corresponding circular pattern on your retina. Your brain interprets the signals from the retina as an image of the object.
The simplest way to do this is if the light rays travel directly from the object to your eye, and the lens of your eye bends the diverging rays so they land in the right pattern on your retina.
However, there are other ways to manipulate the light rays so that they can arrive at the lens of your eye and be bent to form an image on your retina.
One way is to insert a concave lens between the object and your eye. The lens simply increases the divergence of the rays. Your eye can still bend them to fall on your retina. This tricks your brain somewhat, since the bending of the light by the concave lens makes the light rays appear to have come from a point between the object and the concave lens.
Another way is to use a convex lens. This bends the light rays so that they converge to a point and then spread out again before reaching your eye, where your eye's lens bends them again to focus on the retina. This tricks your brain in a different way, since the brain now interprets the light rays as having come from the point where they were converged by the convex lens.
In all the cases I have described, what you actually see is an image on your retina. That can be:
1: light coming directly from the object itself, being bent for the first time by your eye.
2: Light appearing to come from a point between the object and an intervening concave lens.
3: Light appearing to come from a point between a convex lens and your eye.
In 3, we say that a real image is formed by the convex lens. What we mean by that is that the lens bends the rays of light so that they converge in such a way that their spatial arrangement corresponds to the relative positions of the points at which they left the object.
In 2, we say that a virtual image has been formed, because if you trace the rays backwards, they appear to have come from a source that again resembles the original object, although that is just an illusion.
The reason why convergence, or apparent convergence in the case of a virtual image, is important, is that your brain seems to interpret images by in effect tracing back along the path of the light rays to their source. When you see a pink circle what you are seeing is the collection of points from which the light rays hitting your retina appear to have originated. If the light rays are made to converge to a point and re-diverge by a convex lens placed in from of the object, then to your brain it seems that the rays originated from that intermediate point of convergence, so that is what your brain 'sees'.
