Let's suppose we have a spring fixed to a wall, attached to a block and nothing is said about the forces acting on the block or spring.  It's known that if the block is displaced in the positive direction, the work done **on** the spring will be equal to $\frac12 kx^2$ and the work done **by** the spring will be $-\frac12 kx^2$.  How can the work done be equal and opposite when to displace an object the forces required should be unequal?