No mater the inertial referential: The pain is linked to the energy dissipated by the change of speed of the 2 objects in any inertial referential.
Remarque 1: a part of the energy can be absorbed by the ball by deformation or heating. So a soft ball would make less pain.
Remarque 2: The pain depends on what is static behind you and what is static behind the ball. Example in a squash court: you at 50km/h or ball at 50km/h will not produce the same pain further after the ball contact.
More over if the thing behind you or behind the ball is really heavy, it will produce gravity attraction. As attraction is proportional to the square of the distance with the attractive object, you and the ball will not undergo the same acceleration. So speed will be altered.