Riemann developed Riemannian geometry for purely mathematical, geometrical, logical reasons. But after he did so, he went on in print to make theoretical physics speculations. (To be specific, that the curvature of space was caused by the matter occupying it.) They were not much understood at the time, but Einstein's work verified those speculations, and also filled them in considerably, giving them greater specificity, extending them to four dimensions instead of three, an indefinite metric instead of a definite one, and other developments and alterations. Weyl gives an account of this in his book, Space-Time-Matter.
Although Maxwell used the previous physics of Faraday and others, there was one small part missing which he supplied himself purely by mathematical analogy with their work, after having mathematized their work. In a small way, then, this is also an example. But more importantly, although the earlier scientists did indeed have the idea of the field and current, it was Maxwell's examination of the maths that led him to the physical idea of an electromagnetic wave. This is huge: a wave without anything material which it is a wave of. (It took a long time for physicists to accept this.) Our modern physical notion of a wave came from this maths.
Less clear is Hamilton's purely mathematical examination of the relation between geometric optics and the wave theory of light, which he then extended to Newtonian Mechanics. But I think this counts as well: the mathematical structure of Hamiltonian Mechanics and its duality between wave theory and Newtonian particle theory is quite consciously what led Schroedinger to discover Wave Mechanics (in the Quantum Theory) and for a couple of years no one knew what the physics of the wave function was: they worked with $\psi(x,y,z,t)$ based purely on the Hamiltonian maths, and only later did Born discover the accepted physical meaning of this wave. So I think this is again an idea of the maths, the wave function, leading to the discovery of new physics later, by Schroedinger and Born.
Even less clear is Hilbert's discovery of linear operators and their spectra. He named the 'spectrum' of a linear operator the spectrum deliberately because it looked like the atomic spectra then being studied, but put in a footnote that of course this was only a figure of speech, an analogy. Later, Born (a physicist who was not exactly his student but someone who had worked with him) pointed out to Heisenberg that this was the maths that described Heisenberg's quantum mechanics. But this is not exactly the physics (in the sense of physics ideas) growing out of the pure maths. It is rather the mathematicians having invented, in advance, and purely for their own reasons, exactly the maths needed for the physicists to formulate the physical law with.
This has happened over and over, as pointed out in some of the other posts, but is may not be quite what the OP was asking about, which seems to be whether someone, for purely mathematical reasons, discovers a physical idea, a physics concept. Hilbert did not have any such, nor did Levi-Civita. But Riemann did.
