I mean, we can also feel non-contact forces, we just usually are not in a good situation to feel them. Like, if we were standing on a good mirror in a "heliosynchronous" orbit above a star, we'd presumably feel the tidal force stretching us. There's nothing magical about contact per se.
What do you feel?
You feel stimulations of your nerve fibers, of course! But what stimulates them? Well, your tissues and organs have to stretch or compress, mostly. So, that means that you can't feel a constant acceleration of every part of your body, but only when part of your body is feeling some acceleration that another part is not.
The four fundamental interactions/forces
Physicists today are pretty comfortable lumping the interactions of the world into six fundamental fields usually arbitrarily grouped together as four fundamental "interactions":
- the gluon field, or strong nuclear interaction, keeps protons sticking to each other;
- the W and Z fields, or weak nuclear interaction, describes radioactivity;
- the photon field or electromagnetic interaction covers everything else at the atomic level including all of chemistry and why you're not falling through the floor;
- the Higgs field, which is subtle enough at our present temperatures that we don't usually refer to it as an interaction, mostly gives a little extra mass to some particles.
- the metric tensor field, or gravitational interaction, covers masses warping space to draw each other nearer; and
These are not yet all unified in one cohesive theory. There are three problems: (A) our understanding of 1-4 has been in terms of a "quantum field theory" called the Standard Model, but we haven't been able to generate a good quantum field theory approach to model gravity; (B) moreover 2 and 3 happen to "unify" nicely into an "electroweak" interaction (and this theoretical nicety is how we discovered 4), but there are too many proposals to work 1 into this to great a "grand unified theory"; (C) our model for 4 lacks an important property called "asymptotic freedom" that allows us to extend the model to arbitrarily small distance scales; we can fix this, but it's ad-hoc and doesn't give us any insight into quantum gravity, which is predicted to become important at small distance scales anyway.
But in any case, we have a patchwork of theories which seems to cover all known experiments, and resolving the tensions between the theories to create a "theory of everything" is an open research problem.
You are a small electromagnetic blob.
So let's apply these two aspects to each other! We know all the things that happen and we know what you feel, what sort of thing are you?
The first thing is that you're not one atom or even a molecule, you're a blob in the sense that you have tons and tons of atomic nuclei in you. The stuff that you contain does have nuclear interactions occasionally, but usually this is such a small effect that except for, say, radiocarbon dating when you die and are fossilized, or if you are irradiated by nuclear fallout, we're not concerned with the nuclear interactions. These don't push your limbs apart, certainly. Remember, we're looking for accelerations of one part of your body relative to another. (1) and (2) above are out. Similarly, the extra mass from the Higgs field is important but you never feel it as a force, so (4) is out too.
But you are not a big blob, from the point of view of gravity acting at densities similar to the matter on Earth. From gravity's perspective, you are a small blob. You aren't the size of a moon or a planet or a star. This is just saying: the tidal force across your body is mostly negligible in the everyday world, the gravitational acceleration mostly accelerates every atom in your body the same way. So that is why you do not feel gravity. Maybe some day we will be able to generate strong enough gravitational radiation that you could feel it, or get you close enough to a star (perhaps a neutron star?) to make you feel tidal forces. But it's not easy to feel such things on Earth.
Instead, your biophysics is dominated by biochemistry: you are a small electromagnetic blob, and most what you think of as "you", taking up "space", is actually the wavefunctions of electrons bonding together nuclei into the chemical compounds that make up your cells and you. That's what you are.
And I've got some bad news for you: to a very close extent, you have no net charge. The exception might be when you have some static electricity on you; we can make your hair stand on end with that sometimes. The problem there is that too much static creates, well, static -- sparks that jump from you to the rest of the world, which is also not very charged. And you would need a lot of that charge if we wanted to throw you around with the electromagnetic field.
That's not quite true, because there is also electromagnetic radiation. Electromagnetic radiation is awesome. Unfortunately again: photons have an energy-to-momentum ratio of $c,$ which is a huge number that means that you notice that radiation as warmth (or sometimes DNA damage for wavelengths smaller than blue light) rather than as something which pushes you. (I vaguely remember calculating that even if you had a coat which reflected 99.99% of light I couldn't smack you backwards with a light cannon without boiling you alive. It's something of those orders of magnitude.)
So due to our smallness (gravitationally speaking), our blobbiness (nuclearly speaking), and our neutral electric charge, our principal manner of stretching is electromagnetic contact forces. Those are just all that's left when "static EM fields at a distance, dynamic EM fields at a distance, gravity at a distance, nuclear interactions" are ruled out.