From a circuit theory perspective, recall that the product of voltage and current is power:
$p(t) = v(t) \cdot i(t)$
Also, for the inductor:
$v_L(t) = L \dfrac{d}{dt}i_L(t)$
So, there is only a voltage across an inductor when the inductor current is changing with time.
It follows that power (time rate of change of work) is supplied to or delivered from the inductor when the inductor current is changing with time.
But, the magnetic field threading the inductor must be changing with time if the inductor current is changing with time.
Finally, recall that a changing magnetic field induces a non-conservative electric field and thus an emf which can do work.
Remember, for a constant current through an (ideal) inductor, there is no associated power as there is only a steady magnetic field and thus, no induced voltage.