I do know something about the history of the problem, but I don't know the answer. The first reference is
- S. Lloyd, Almost any quantum logic gate is universal, Phys. Rev. Lett. 10, 346–349 (1995).
which gives a "proof" of this... but it isn't strictly correct. I don't know what exactly was wrong with it, because I never looked at it in detail. It doesn't seem to be on the arxiv, either. The qubit case was actually proven in this paper:
- D. Deutsch, A. Barenco and A. Ekert, Universality in quantum computation, Proc. Roy. Soc. London A, 449, 669-677 (1995). arXiv:quant-ph/9505018
Looking through it, they seem to write what you say as a conjecture on p 6. It's the "canonical" universality reference AFAIK.
This next paper discusses the qudit case, but from the perspective of Hamiltonians (generators of the unitary group). I think it might contain the statement that you're after, but don't see the paper on the arxiv (annoying), so I can't check at the moment.
- N. Weaver, On the universality of almost every quantum logic gate, J. Math. Phys. 41, 240 (2000).
Also, this next paper is at least related (though it is again about Hamiltonians), so the background section might contain a reference to the theorem that you want.
- A. M. Childs, D. Leung, L. Mancinska, M. Ozols, Characterization of universal two-qubit Hamiltonians, Quant. Inf. Comp. 11, 19-39 (2011). arXiv:1004.1645
Off the top of my head, I don't see any reason why the Hamiltonian case should be very different from the unitary case, but I suspect this is the trouble with the Lloyd paper, so I would tread carefully if I were you.