The current flows "forever" (for many practical purposes) until, indeed, there is a closed loop.
A common way to generate a persistent current is to have a solenoid driven by a current source. The two ends of the solenoid are then joined by a superconducting link which can be heated by an electrical heater. When the heater is on, the superconducting link works as a normal conductor and the current crossing the solenoid is that coming from the current source. When the heater is turned off, the link becomes superconducting and the current in the solenoid will flow through this link. At this point, you can slowly turn off the current generator but the current in the superconducting solenoid will continue to flow persistently.