I have derived a quantum quDit gate that is implemented by photon Fock state inteference. It turns out to be non-unitary. I thought I must have made a mistake, so I have checked several times. I know a qubit gate must be unitary, but can a quDit gate be non-unitary?
No, a qudit gate must also be unitary, just as all quantum gates. What can be done, however, is to probabilistically implement a non-unitary gate (in fact, any non-unitary operation). If your gate is e.g. only implemented conditioned on the detection (or non-detection) of a photon, then this is possible.