The brain does flip the image horizontally, too, but talking of "flipping" in the first place gets a bit into pseudoscience, since the brain doesn't really do any "flipping" of images or have any a priori awareness of how the retina is constructed - it simply learns by experience (as a baby) that moving the left hand leftwards, produces a change in certain inputs, and that all things that lie on the left-hand side of the visual field correspond to these inputs - inputs that only adults know lie on the right-hand side of the retina. To the brain, those right-sided inputs in the retina are the left-hand inputs, without any correction needed.
That's also why the "blind spot" at the optic nerve is invisible, because the brain simply has no inputs from that area, and two-eyed people can always see the full field of view, so nothing is routinely seen to "disappear".
You can see your entire fingertip disappear if you look at it steadily off-centre, out of one eye, with the other closed, from about a foot away - and if you wiggle the fingertip, it cannot be seen despite being in front of you.
But we so rarely allow (or need) those circumstances with our eyes to coincide that it rarely has any bearing and is simply unnoticed.
Edit: It's quite curious actually. At a little over a foot away and looking through my left eye only, I could wiggle my left index finger significantly in a beckoning fashion, and I cannot see my finger at all (my palm appears to have 3 fingers and a thumb), although I can see the changing shadow that the movement casts on my thumb (because of the window to my right).
When my finger leaves the blind spot, it appears to fade in from the background, and the background (which is only vague anyway when viewed off-centre) is interpolated from my wider field of view.
For anyone interested in where to find the blindspot in the field of vision, the blind spot in my left eye roughly coincides with me stretching my right arm out in front of me, with my palm facing the floor, and the fingers of my right hand splayed, and if I keep the central focus my left eye on my right little fingernail, my right thumbnail is in the blindspot and cannot be seen.