Playing a vinyl "LP" implies a 33 rpm motion and 30 cm diameter. The highest frequency recorded will depend on the track velocity and the size of the needle.

30 cm diameter implies a 100 cm track length (roughly - less as you move further in) traversed in about 2 seconds - 50 cm / second. The radius of the needle is specified in the standard as less than 25 um. As the frequency increases, the ability of the stylus to follow the groove is limited. 10 kHz would have a "wave length" in the groove of 50 um, which is starting to be at the limit.

Mechanically then, LPs cannot record much beyond 10 kHz without using a smaller-than-standard stylus. In fact the cutoff of frequencies recorded today is around 24 kHz.

The smaller the amplitude, the higher the frequency you can record; and the same is true for the stylus. There need not be a hard limit: but existing equipment was optimized to work with the audio range of today's humans so expect to have to design a new stylus, amplifier, equalizer, ... If you want to extend it.