Interesting. In your title you say "background noise." So I was going to suggest the sea shell effect - putting a sea shell to your ear you think you hear the sea, even if it is far away.
If you did think you heard music, that could have been an illusion. Your brain is wired to look for patterns - see animal shapes in the clouds, hear people talking (or music playing) when the vacuum cleaner is on, find co-incidences in the events of life, etc. From among external noises resonating in the earphone and funnelled into your ear, your brain picks out familiar sounds of the kind it expects to hear.
But if it was a recognisable, continuous tune which stops after a predictable time and does not come back, there must be some other explanation.
An alternative explanation which I am less sure about, but which is mentioned quite often with speakers, is that part of the earphone (possibly the wire lead) is acting as an antenna, picking up radio waves.
The earphone contains a few capacitors which get charged up during use. When you disconnect power the charge on the capacitors slowly drains away at a rate depending on what resistors they drains through. (Recall that RC is the time constant.) The leaking capacitors could be powering the amplifier in your earphone, amplifying the pick-up of radio waves, until they have fully discharged.
During normal use the pick-up noise ("radio interference") might still be there but insignificant compared with the music you are listening to.
This explains better why it sounds like a radio station (because it is a radio station) and why it persists for only about 1 minute after disconnecting the power (because that is how long it takes the capacitors to discharge).