So far what I had understood about potential energy is that it is defined for a system of particles (at least two particles) with forces acted by the particles on each other of same magnitude and opposite in direction as, $$-W_\mathrm{conservative} = ΔU.$$

For instance in case of gravitational potential energy, we only consider work done on small body as there is negligible work done on the earth. But in cases such as the following example, the concept is not completely clear. Consider a massless spring, the left side of which is attached to the wall, and there is a block attached to the right side of the spring. If we consider the block $\cup$ spring as our system, then the the net work done by internal forces would be zero. So what is increasing the potential energy of the system as we stretch the spring?

As per one of the answers to this question, it seems we need to consider the earth to be a part of the system and the spring just stores potential energy similar to the gravitational field while not really being the part of the system.

Is this reasoning correct? If not, please help me clarify the concept.