I am basing my answer on an answer to a similar question. While Quantum Mechanics correctly predicts this phenomenon, I'll give a Semiclassical approach.
So, the atoms in a diamagnetic material have all their electrons paired, which means that both their spin magnetic moments and orbital magnetic moments cancel each other, leaving each atom with zero net moment.
When an external field B is turned on, there is a brief lapse where B is changing in time (from off to on), so by Faraday's Law of Induction, an electric field appears and forces the electrons to change their orbital momenta, as to create a net magnetic moment anti-parallel to B (Lenz's Law).
Eventually B reaches a time-constant value, so there is no more induction. However, as long as the electrons maintain their orbital momenta, the magnetic moment will continue existing. You could imagine the electrons semiclassically as a "perfectly conducting current loop", they keep looping by their own inertia, so continue producing the (originally induced, but now "permanent") magnetic moment. This is something that never happens in real macroscopic circuits because there is always some resistance and kinetic energy eventually dissipates away.
If we switch off B, there will be another lapse where B changes from on to off, so Faraday's Law/Lenz's Law will act exactly opposite of what happened first. The result is that the orbital momenta are returned to their original state, and the magnetic moment disappears.
Because of this, we say that diamagnetic materials "show an induced magnetic moment in the presence of a constant B field, which goes away when the B field does". But this phrase hides a bit more complexity.
Note: While magnetic moments will rotate in the presence of B fields, in order to be repelled (and thus levitate), the B field also needs to be non-uniform in space.
