Potential energy is defined as negative work. If I do work on a rubber band by pulling it back, it stores the work I did on it as elastic potential energy, meaning as a result as I let go of one end of the rubber band the other will convert the potential energy I gave it into kinetic. But what implies that this stored potential energy is going to be converted into kinetic energy in the first place? This may be so obvious its dipping into ridiculousness, but I can't explain it other than "it just will, it's been pulled back and potential energy can convert to kinetic energy if certain conditions are met, like letting go of the band."
Yes it does look ridiculously obvious. Perhaps because you are preoccupied with abstractions like potential energy instead of tangible effects like forces.
You are pulling the band to the right, the rubber band is pulling itself to the left. The more force you use, the more the band pulls back. When you let go the band doesn't stop pulling; it pulls itself to the left, initially with the force you were using just before you let go. The band has a small mass, so it accelerates quickly.