The notion of temperature is all about how the equilibrium an otherwise isolated system shifts when the system's internal energy changes. So you do not need to worry about whether this internal energy is kinetic, potential, whatever.
Actually the temperature is not quite the ensemble average kinetic energy. Your statement is true for an ideal gas and also approximately true for many substances at highish temperatures (usually tens of kelvins and above). But you can see perfectly well for yourself, given your excellent question, that there's a problem with things like photons.
The most general definition of thermodynamic temperature for a system at thermodynamic equilibrium (temperature can only strictly be defined for systems at thermodynamic equilibrium) is in fact:
$$\frac{1}{k_B\,T} \stackrel{def}{=} \beta = \frac{\partial\,S}{\partial\,U}$$
where $S$ is the system's entropy and $U$ its total internal energy, whether it be kinetic, potential, whatever. I like to think of this information theoretically: you think of the system as an isolated box, and you shove a bit of energy into it and ask how much the informational content of its microstates changes in response. To explain further: photons (or a system of quantum harmonic oscillators) actually give a nice example here: if you heat them up, i.e. give each oscillator more energy, it can access higher and higher energy states. Therefore, you need a bigger "alphabet" or more bits to specify the state of each oscillator. You can also see how this ties in with the approximate notion of temperature as a mean energy. For most particles, the higher their energy, the more bits you need to describe their state. A quaint name for $\beta$ is sometimes the "perk": how much a system rouses or "perks" up in response to its "eating" energy.
But temperature can also be negative! This happens when the constituent particles have a strict upper limit to their energy levels. As you shove more and more energy into the system, the particles become more and more certain to be in their highest energy state as this is the only way for the system to take on more energy. There is a limit to what it can swallow. When you reach this upper limit of total energy, everything is certain to be in its highest energy level and the entropy is nought! As you approach this level, the system's entropy decreases as you shove more energy in, so the partial derivative, and the temperature, is negative. I do some more detailed calculations around this idea in my answer here.
Although I find the information theoretic ideas most intuitive, the definition $\beta = \frac{\partial\,S}{\partial\,U}$ above goes right back to Carnot and Clausius themselves. It is a differential form of their original definition of temperature in terms of efficiency of heat engines linking a reservoir whose temperature is to be measured to a "standard" reservoir whose temperature is by definition unity. See my answer here for more details.
Lastly, let's go back and apply all my banging on to your original system - a system of photons. You simply apply the ideas of the canonical ensemble to a system of oscillators with evenly spaced energies, and find out what the maximum likelihood microstate distribution, given an assumption of constant total system energy, is. At this point you find that for the ensemble of quantum harmonic oscillators, the mean oscillator energy is:
$$\left<E\right> = \frac{\hbar\,\omega}{2}\,\coth\left(\frac{1}{2}\,\beta\,\hbar\,\omega \right)$$
Here $\beta$ is the "perk" or reciprocal temperature we defined above. It is a Lagrange multiplier that one defines in the course of finding the constrained maximum likelihood distribution. The Shannon entropy (per oscillator) when the maximum likelihood distribution is reached is:
$$S = -\sum\limits_{n = 0}^\infty p(n) \log p(n) = \frac{\beta\,\hbar\,\omega\,e^{\beta \,\hbar\,\omega}}{e^{\beta\,\hbar\,\omega}-1} - \log \left(e^{\beta\,\hbar\,\omega}-1\right)$$
so the thermodynamic temperature is then given by (noting that the only way we change this system's energy is by varying $\beta$):
$$T^{-1} = \partial_{\left<E\right>} S = \frac{\mathrm{d}_\beta S}{\mathrm{d}_\beta \left<E\right>} = \beta$$
as foreseen (I knew the answer of course, so no magic here!).
So a gas of photons definitely can have a thermodynamic temperature. What's weird and unwonted about this case is that they cannot interact directly, so there is no obvious way for them to thermalise. In practical, natural systems they thermalise by interacting with the stuff around them, and this is probably why the CMBR is indeed observed to have reached the maximum likelihood, thermodynamic equilibrium distribution.
Appendix: Two Equivalent Interpretations of Temperature
One can show that the reciprocal temperature $\beta$ defined as $\beta = \frac{\partial\,S}{\partial\,U}$ is the same as the Lagrange multiplier, i.e. the multiplier in the exponential arising in the Boltzmann distribution for much more general systems of statistically independent particles, not simply for harmonic oscillators.
The Boltzmann distribution for the probability of finding, in a system of statistically independent particles at thermodynamic equilibrium, a constituent particle in state $i$ is:
$$p(i) = \frac{1}{\mathcal{Z}(\beta)}\,\exp\left(-\beta\,E_i\right);\quad \mathcal{Z} = \sum\limits_k \exp\left(-\beta\,E_k\right)$$
where $E_i$ is the energy level of state $i$ and $\mathcal{Z}$ is the partition function that normalises everything so that the probabilities all sum to unity.
Thus the Shannon entropy per particle is:
$$S=-\sum\limits_k p_k\,\log p_k = \sum\limits_k (\beta\,E_k + \log\mathcal{Z}(\beta) - \log q_k)\,p_k = \beta \,\left<E\right>(\beta) + \log{Z}(\beta)$$
Then, suppose $S$ varies as a result of the variation of $\beta$, $\left<E\right>$ and $\left<q\right>$. We have:
$$\mathrm{d}\,S = \left<E\right>\,\mathrm{d}\,\beta + \beta\,\mathrm{d}\,\left<E\right>+\frac{\mathrm{d}\,\mathcal{Z}}{\mathcal{Z}}$$
Now we note that $\mathrm{d}\,\mathcal{Z} = -\left(\sum\limits_k q_k\,E_k\,e^{-\beta\,E_k}\right)\,\mathrm{d}\,\beta = -\mathcal{Z}\,\left<E\right>\,\mathrm{d}\,\beta$, so that we are left with:
$$\mathrm{d}\,S = \beta\,\mathrm{d}\,\left<E\right>$$
which, when scaled up by the number of particles in the system, is simply $\mathrm{d}\,S = \beta\,\mathrm{d}\,U$.
In a system with a negative temperature, therefore, the higher the a state's energy, the likelier it is to be occupied.
