No and it's exactly the second law of thermodynamics that applies (in the Clausius formulation). You answered your own question. If you want more detail, assume that the solar panels have the efficiency of a Carnot machine. What's the efficiency then? 0% because there is no temperature difference.
While solar cells are not mechanical engines, they are still subject to the laws of thermodynamics. One can analyze the details in terms of incident photon energy and momentum, electron-hole energy and momentum and the necessary coupling to the crystal lattice which has a non-zero temperature. The result would be the same as with all such analyses: no system can exceed the efficiency of a Carnot machine.
I think the conceptual problem that many are having (I had it, too) stems from the fact that we are used to looking at the idealized absorption of a single photon of just one wavelength in an ideal cell at 0K and we extrapolate the physics of that to black body radiation and hot cells.
A more refined model of how solar cells work for thermal radiation is given by the Schockley-Queisser limit https://en.wikipedia.org/wiki/Shockley%E2%80%93Queisser_limit. This also explains why we are "stuck" at approx. 20% efficiency with the current generation of single junction cells. It's not a technical problem but a direct result of thermodynamics!