How can we be sure that a new frequency standard is better than the old one? Lets assume for this question that at one point in scientific history everyone was running along well with their cesium frequency standards, and someone has a brilliant idea and builds an active hydrogen maser frequency standard. Calculations show that it should be better, he builds it, compares it to his venerable cesium frequency standard, and there is indeed a difference, lets say for the sake of this question, an allan deviation of $10^{-14}$ for 1000s gate time.
But in the end, this can only mean that one of the two is better, how can we be sure it is the new hydrogen maser, and not the old cesium beam?
 A: Your question is basically, "If I have a type A clock which runs faster than a type B clock, is A correct and B slow, or is B correct and A fast, or are both wrong?"
The question is answered by making several of each type and comparing them to others of both types. There are very sophisticated techniques available for detecting deviations on different time scales. Testing at different temperatures, for instance, will determine which type has better temperature stability, etc. 
And keep in mind that one obvious quality has no meaning: absolute accuracy. There is no such thing. All that can be measured is consistency and stability.
A: In my humble opinion, what determines if one frequency standard is better than another depends on our ability to make precision measurements using that standard.
As an example, take the definition of the meter. Originally the meter was defined using the famous physical meter rods. Unfortunately because these are physical objects they are sensitive to changes in temperature, pressure and who knows what else. This restricts the accuracy with which one can know how long a meter is.
Our current definition of the meter is instead tied to fundamental constants of nature such as the speed of light and the caesium frequency mentioned above. This way a more precise measurement can be obtained for the length of one meter.
I presume the same requirements holds for the choice of frequency standard, be it the caesium frequency or the maser frequency. 
A: Build two of each and beat them against each other. If two of them stay closer together for longer then they make a better frequency standard, since presumably I can then go tell my friends "hey, build a hydrogen maser" and they'll end up with a clock that keeps time with mine. Or at least, a clock that does it better than my other friends who have built cesium clocks. It therefore meshes well with what we mean by a standard.
