The "actions" of Newton's Third Law are Forces, not distances, velocities or even momenta. Your distance from the wall and the distance that the ball rebounds have nothing to do with each other under the 3rd law. The "Opposing Force" when you throw the ball is the force that is pushing you back against the ground that you are standing on. The Opposing Force when the ball bounces off the wall is the force that is pushing the wall against its foundations.
If you stand on Teflon ground wearing Teflon shoes, you will not be able to throw the ball as far, because you cannot get the usual opposing force from the ground and you will be pushed back a little.
If the wall has no foundations (and is thin and also not very heavy) then the ball will not bounce as far and the wall may fall over, again because you lack the usual opposing force that was being supplied by the foundations.
All the Third Law really says about your ball bouncing off the wall is that the implicit forces applied to your ball and the those applied to the wall are the same, but opposing.
What the difference in their bounce is really about is a difference in how much energy is lost (dissipated) by one ball versus the other. The cloth ball dissipates more energy than the rubber ball, probably by redirecting it into changing the shape of the cloth ball.