What is temperature? There are very formal mathematical answers to this question. However, the best answer I have encountered in my six years of physics education was in my original thermodynamics course my second year, in Schroeder's Thermal Physics, pages 85-91. However, my understanding has evolved with exposure to probability and information theory.
Temperature is unambiguously defined as "how much a system's entropy changes when that system's energy changes."
Thus whatever understanding of temperature one wishes to obtain is fundamentally limited by their understanding of what entropy is. Entropy is a measure of how much information (in bits, since we're on computers right now) is required to know the state of a system.
A system's state (really, quantum state) is everything that can possibly be simultaneously known about a system. Once you know everything there is to know about a system, you have determined its state.
Entropy is equivalent to the expected number of yes/no questions minimally required to determine the state of a system. Please note the word "expected" (which means average), and the word "minimally"(which means asking the best questions you possibly can).
You probably have never heard this definition of entropy, but this definition is actually completely correct, except in physics we multiply this number by $k_b ln(2)$ (a number) merely for historical reasons. So whenever you read entropy, you should try thinking expected number of yes/no questions. If you don't know if a coin is heads or tails, the entropy is 1 binary question: "Is the coin heads?".
There is a simple law saying the expected number of yes/no questions required to determine the state of a closed system can never decrease. This is known as the 2nd Law of Thermodynamics. It's a cool law. And when entropy is defined as the expected number of questions, it's an exact law that always holds. It even holds for Maxwell's Demon.
The expected number of questions to determine the state of a closed system can certainly increase. And it certainly will, until it hits a limit. A system that has hit this "limit of unknowability" occupies every possible state with equal probability, and I call this system ergodic. This always happens if you wait long enough, thanks IMO to the mathematics of markov chains (every closed system is necessarily an irreducible, ergodic markov chain that approaches a stationary distribution). This is called the ergodic hypothesis in physics.
Consider two ergodic systems, one high temperature and one low temperature.
When a system has high temperature, it means that small changes in the system's energy cause large changes in the system's entropy (in fact, this is the definition of temperature). Thinking about entropy as the expected number of yes/no questions, this means your going to have to ask a lot more questions to determine the state of the system if you add a little energy.
When a system has low temperature, it means that small changes in the energy of the system do not change the entropy of the system very much. You're not going to have to ask significantly more questions to determine the state of the system if it has a little more energy.
Now consider the combined system, closed off from the rest of the universe. The 3rd Law places a restriction on the expected number of yes/no questions to determine the state of the combined system. Consider what happens if the systems can exchange energy (and only energy!).
If energy is not exchanged between the low temperature and high temperature systems, then the expected number of questions required for the whole system $N_{1+2}$ is just the sum of the expected number of questions for each subsystem: $N_{1+2} = N_1 + N_2$.
However, what happens if the two subsystems can and do exchange energy? The 3rd Law says that, whatever happens, the expected number of questions required to determine the state of the combined system cannot decrease.
If you know that more energy flowed from the high temperature system to the low temperature system (which it certainly can, energy flows randomly), you know from the definition of temperature that the number of questions required to determine the state of the combined system has decreased, in apparent violation of the 2nd Law: $N_{1+2} < N_1 + N_2$. However, this knowledge about "backwards flow of energy" cannot be obtained without asking a certain number of questions $N_q$ of the system: the exact number required by the 2nd Law $N_{1+2} \geq N_1+ N_2 + N_q$.
On the other hand, if all you know is energy exchange is occurring in this combined system, from the ergodic hypothesis the expected number of questions you'll have to ask is only increasing, rapidly approaching the ergodic limit. This requires that energy flows on average (randomly) from the hot thing to the cold thing. And the ergodic limit is when the hot thing and the cold thing are the same temperature.