The question overloads the word "system", so I am going to plow ahead assuming I know what you mean. but first:
When addressing these conservation/symmetry type question, it's really important to abstract away real world concerns.
So if I am looking at a frictionless ball rolling on a parabolic surface, that is my system.
That it is frictionless means I've abstracted away molecules, that's $N_A$ degrees of freedom that I am not considering.
Of course my surface exists in a uniform gravitational field which means I've abstract away "the earth", and don't worry about its momentum and energy. (So in my system, momentum is not conserved because the external $\vec g$ breaks translation symmetry).
I can also abstract away other spatial dimensions, which is very common in physics problems.
So in your question, I'm assuming the first usage of "system" means some sub-system that is part of your actual system, and actually...idk why you've introduced "external system"...does the mean external to the object? I'll go with that.
So here is my hypothetical system: I have a system. It's 2 dimensional. It has a sub-system ($A$) that does work on an object $B$, and $B$ gains kinetic energy.
For now, the entire thing exists in a uniform gravitational field. This could be from the Earth (in the approximation that Earth is fl.., uh, let's just say: not-round), and if that bothers you, then we're in an elevator in deep space undergoing constant acceleration.
System $A$ could be an ideal spring, $k$, attached to the wall. That the spring is ideal (no mass, no damping) means I've abstracted away microscopic DoF. The attachment to the wall means I've abstracted away "recoil".
The spring starts compress by $L$ away from equilibrium, and is attached to $B$.
OK, start the system:
The subsystem released it's potential energy:
$$ U = \frac 1 2 k L^2 $$
so that $B$ starts moving at $v$, with:
$$ T = \frac 1 2 m_B L^2 $$
So in this system, the energy was stored in the spring. Since it's an ideal spring, that's as deep as it goes.
If it's a real spring, then it has to have mass and damping. The potential energy is stored in the interatomic forces of the material making the spring, but there is no way to transfer all that energy to $B$, some will be left in the vibrating mass of spring (not lost, since the spring is in my system), and some will be lost to thermal/vibration modes of the spring's atoms (that energy is lost, because those DoF are not part of my system).
Or, system $A$ could be bulk TNT:
in which case AI tells me: The energy is stored within the arrangement of atoms in the TNT molecule, particularly in the nitro groups attached to the toluene ring.
System $A$ could be a charge capacitor that does something to make $B$ move, then the energy is stored in the capacitor. Where that energy is in charges at voltage ($\frac 1 2 QV$) or electric field ($\frac 1 2 \epsilon_0 E^2$) is up to you.
System $A$ could balance $B$ on an un stable equilibrium (e.g. a ball on the tip of a pencil (length $L$)), in which case the energy is stored in the potential energy of the ball, $mgL$. Note that the gravity is external to whole system, so that means I am considering a universe where energy is position dependent, and that breaks some conservation laws (see: Noether's theorem).
You can think of all kinds of ways to store energy in subsystem $A$: gravity, electromagnetism, the strong force. I am not sure how to store energy in the weak force, though.
I suppose if the subset $A$ is a neutron, and the object $B$ is an electron...I mean the weak force is just a trigger, and the energy is stored in mass differences of the particles involved.
tl;dr The correct answer to your question is: anyway you want to store energy, but the take away is: define your system abstractly, and know what that means. Don't get hung up on reality, esp. if the system is "frictionless" or "ideal", and external fields ($\vec g$, $\vec B$, whatever) are external, but not part of your system; however, that means your system can break cherished conservation laws like $\vec p$, $\vec L$, and if your system is time dependent: $E$.
