13
$\begingroup$

Analysis of data is integral in bridging the gap between theory and experiment. How much do the results of the analysis depend upon the choice between Bayesian and frequentist methods?

For instance, consider experiments in particle physics. This paper by Louis Lyons states that particle physicists use a hybrid approach to analysis (frequentist hypothesis testing and Bayesian parameter estimation). If a purely Bayesian/Frequentist approach were applied, would there be a difference in the results of the analysis? What implication would any difference have on the interpretation of data?

|cite|improve this question
$\endgroup$
  • 1
    $\begingroup$ (Disclaimer: theoretician speaking) I would be perfectly happy if the experimentalists used a purely Bayesian approach, as long as they don't let the theorists set the priors. :) That said, I don't know that there can be any short answer to this question... $\endgroup$ – Michael Brown Aug 1 '13 at 5:17
  • $\begingroup$ @MichaelBrown Why shouldn't theorists set priors? A bit strange thing to hear from a theorist. :) With the question I'd be happy with a single example as opposed to a comprehensive analysis of all of experimental Physics. $\endgroup$ – Comp_Warrior Aug 1 '13 at 12:56
9
+50
$\begingroup$

If there are enough data and the prior is not completely unreasonable, the frequentist and the Bayesian approach give essentially the same answer. This is related to the central limit theorem.

If data are fairly scarce, the two approaches may differ a lot. In this case the Bayesian approach is far preferable but only if the prior reflects true prior knowledge and not just prejudice. (With a drastically wrong prior and limited data, the Bayesian approach tends to reaffirm the prejudice, and hence will be far worse than the frequentist result.)

To see this, take a prior and assume that just one observation. It is clear that the Bayesian outcome is just a small change of the prior. So if the prior was appropriate (reflected true knowledge), the result is an improvement, while if the prior was bogus (just prejudice), the outcome is as bad. If the number of observations is large, their contribution dominates the outcome, and the result is essentially prior-independent, and is easily seen to agree with the frequentist (maximum likelihood) result.

All this is completely independent of physics. But in statistical mechanics we have a good enough theory that enables us to choose appropriate priors. This is the (only) reason why the maximum entropy principle works there.

|cite|improve this answer
$\endgroup$
  • $\begingroup$ A badly tuned "maximum likelihood" estimator in process control is a good illustration of your last sentence. $\endgroup$ – WetSavannaAnimal Jul 18 '15 at 7:45
  • $\begingroup$ thx for answer but this would be great with some more tangible/ specific example $\endgroup$ – vzn Jul 20 '15 at 21:17
  • $\begingroup$ @vzn: I augmented my answer. You can specialize it yourself to casting dice, with the unreasonable prior being a Dirichlet prior with 100% probability for getting 6, weighted corresponding to 1000 events. $\endgroup$ – Arnold Neumaier Jul 21 '15 at 9:33

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.