Your first statement is mostly true, an object can only reflect light of the wavelengths it is illuminated with. We can have fluorescence though, where light of a given wavelength or range of wavelengths is absorbed and light of a different wavelength is emitted (this is not actually reflection). In the absence of fluorescence, the object will reflect only (part of the) colors it is illuminated with.
You second statement is also mostly true, an object has the color it has because other wavelengths get absorbed. Your conclusion that an object must appear black when it is illuminated by light of a different color than that of the object, is not necessarily true, though it depends a bit on how you define the color of an object (if by color you mean monochromatic color, that would be true). Most of the light we see consists of a mixture of different wavelengths. Different mixtures may actually look the same, because of how the human visual system works. Colors that look different to us, may, and often do, contain common wavelengths.
Finally, if in your linked image you look at the green paprika under white light (which contains wavelengths over the full visible range), you see that it looks mostly green, but two shiny parts look white. In the green part we receive diffusely reflected light, while in the white parts we see a specular reflection, like a mirror. Here the incoming light makes an angle with the surface under which a lot of light gets reflected; in all other direction we lose all light that is not green. If we watch the paprika under a red light, we actually see that it looks mostly black, or at least pretty dark, except in those zones where the reflection is specular and is essentially the same color as the incoming light. This makes it look red.
As for the first question, if we think of colors spanning a broad range of wavelengths, yellow roughly corresponds to middle and low wavelengths, while blue corresponds to high wavelengths. That means that there is little overlap in wavelengths between the incoming yellow light and the light that the paper can reflect, so it will look very dark.