Just like the above image, your situation isn't any different. Some fraction of the outgoing diverging light goes through our eye lens and get converged to the back of the retina (for an healthy eye). Our brain doesn't believe light can be manipulated, so it will trace a light ray straight back to a point of coherence without considering any bend or twist ever happened. This is why even though a virtual image isn't exactly where the brain think it is, it just how our DNA programs our brain to resolve light, and we can't change that because DNA coding is inherent and fundamental.
Even though the light from an object having gone through the lens in our eye converges, the optical sensors in the retina converts the light into electric impulses, it's these impulses that are algorithmically reconstructed back into an image in the brain, and the brain has evolved to translate impulses in certain ways that doesn't magnify nor diminish the formed image, so we see things in the exact size and position they are in reality. You stop seeing things correctly when you put on weird glasses because the brain hasn't evolved the mechanism to translate light through your weird glasses. If you can make a newborn stick to the glass his whole life, the day he decides to take it off, the real view tends to appear warp too. It just down of the inherence of nature's java coding.
Bottom line, a virtual image is a result of the brain's interpretation of light as a straight non- manuvarable thing rather than otherwise.