I have to record this comment by user Velut Luna for its pithy logic:
When we can feel something, we can feel the change of it. We can feel acceleration, therefore we can feel jerk.
which is certainly true, but there is another sense wherein jerk can directly affect our bodies in some cases. Those cases are when one's body is accelerated through the reaction force between the body and a "thrusting" object, such as the seat of a car undergoing acceleration. Our bodies are deformable, and not all parts of them accelerate in the same way: the seat thrusts the parts of the body in contact with it, and these deform. It takes some time for that force to be transmitted through the tissue in direct contact with the seat to the tissue furthest away from the seat. Therefore, accelerations with different jerk as a function of time will give rise to different strains / stresses in the body as a function of position and time.
The same is not true if the body is accelerated by a body force, such as, for example, if it were uniformly charged and accelerated by an electric field. All volumes would undergo exactly the same acceleration so no internal strain would result, whatever the acceleration or jerk or any higher derivative may be. See my answer and the companion answer it references here.