I have often seen diagrams, like this one on Wikipedia for a thin convex lens that show three lines from a point on the object converging at the image. Do all the other lines from that point on the object that pass through the lens converge at the same point on the image?
*Updated question: * to say, "from that point on the object"
No broadly speaking. Yes in a narrow context. Chromatic dispersion guarantees that it is impossible to construct a lens which will be free of caustics and other optical aberrations for all wavelengths. Individually lenses can be engineered to provide nearly arbitrary precision for narrow bands of wavelengths. However any (traditional, i.e. non-metamaterial) optical device is diffraction limited, in that the minimum size $d$ of an object that a wavelength $\lambda$ can resolve is:
$$ d = \frac{\lambda}{2n \sin \alpha} $$
where $n$ is the index of refraction and $\alpha$ the half-angle subtended by the object at the aperture of the device.
What this limit also says about the convergence of light-rays is that any pencil of rays coming from an object of size $ d' \lt d $ will fail to converge at a single point on going through the device.
So the answer to your question is "yes", as long as the device is used within its operating range!
Yes, at least for functional enough lens or mirrors.
As long as the answer is "Yes", the lenses or mirrors will produce an image that is totally sharp. If the answer were "No", a point-like source of light would always look like a fuzzy disk.
The lenses and mirrors and telescopes may be optimized at least for a whole "two-dimensional locus" where the light source may be located to produce sharp images. The shape of the lenses may be constructed so that this condition is satisfied exactly: a real condition (the light ray gets to the right point) has to be satisfied for each distance $y$ from the axis - but one has at least one variable, $x(y)$ (the thickness of the lens), to adjust for each $y$, too. In the first subleading approximation, the shape is always the same: as a function of the vertical coordinate $y$, the thickness of the glass in the horizontal direction goes like $A+By^2$. That approximates a circle, parabola, hyperbola, or anything else, up to errors of order $O(y^4)$.
If you want to totally neglect those terms, it's like neglecting $O(\theta^4)$ terms, quartic in the angle. In this approximation, the angle between the light ray and the horizontal line that is needed for convergence is small and linearly depends on $y$. The linear dependence of the angles on $y$ - how much the direction of the light ray is changed when switching from the air to the glass or back - is equivalent to the quadratic shape of the mirror sketched above (the angle change is a derivative of the shape because this derivative determines the slope of the glass at a given point). This explains why it's not a "miraculous conspiracy": the change of the angle is a linear function of $y$ so the shape of the glass must be a quadratic function of $y$.
In reality, where the $O(\theta^4)$ terms in the shape can't be neglected, the light sources may be away from the plane where the convergence was guaranteed, and in that case, the answer will be "No" and the image will inevitably be fuzzy. However, it's important to note that e.g. telescopes that observe stars "at infinity" may always be adjusted so that all the images are sharp.
Because the shape of the lens actually has two functions, $x_{left}(y)$ and $x_{right}(y)$ to be adjusted, one may actually guarantee that the condition "light rays converge" is exactly satisfied in a whole region of 3D space, at least in some situations.
