Are there any interesting examples of number theory showing up unexpectedly in physics?

This probably sounds like rather strange question, or rather like one of the trivial to ask but often unhelpful questions like "give some examples of topic A occurring in relation to topic B", so let me try to motivate it.

In quantum computing one well known question is to quantify the number of mutually unbiased (orthonormal) bases (MUBs) in a $d$-dimensional Hilbert space. A set of bases is said to be mutually unbiased if $|\langle a_i | b_j \rangle|^2 = d^{-1}$ for every pair of vectors from chosen from different bases within the set. As each basis is orthonormal we also have $\langle a_i | a_j \rangle =\delta_{ij}$ for vectors within the same basis. We know the answer when $d$ is prime (it's $d+1$) or when $d$ is an exact power of a prime (still $d+1$), but have been unable to determine the number for other composite $d$ (even the case of $d=6$ is open). Further, there is a reasonable amount of evidence that for $d=6$ there are significantly less than $7$ MUBs. If correct, this strikes me as very weird. It feels (to me at least) like number theoretic properties like primality have no business showing up in physics like this. Are there other examples of this kind of thing showing up in physics in a fundamental way?