Kolmogorov-Smirnov test vs Chi-squared test What is the difference between the Kolmogorov-Smirnov test and the Chi-squared test?
When should we use one instead of the other?
I was reading this article, and I got confused a lot.
It is hard to get a physics-related answer in the Cross Validated SE section, so this is why I am asking here.
 A: A chi-squared test is used to compare binned data (e.g. a histogram) with another set of binned data or the predictions of a model binned in the same way.
A K-S test is applied to unbinned data to compare the cumulative frequency of two distributions or compare a cumulative frequency against a model prediction of a cumulative frequency.
Both chi-squared and K-S will give a probability of rejecting the null hypothesis. Artificially binning data loses information so should be avoided if possible. On the other hand the chi-squared statistic does give useful shortcuts if you are trying to model the parameters that describe a set of data and the uncertainties on those parameters. The K-S test should not really be used if there are adjustable parameters which are being optimised to fit the data.
Specific trivial example. I measure the height of 1000 people. Let's say they're all between 1.5m and 2m feet. I have a model I wish to test that says the distribution is Gaussian with a mean of 1.76m and a dispersion (sigma) of 0.1m.
So, how do I test whether this model well represents the data? One approach is to construct the cumulative distribution of heights and then compare it against the cumulative normal distribution described using a KS test. However, an alternative would be to put the data into say 5cm bins and then find the chi-squared statistic compared with the model. Both of these would give you a probability of rejecting the null hypothesis. For such a purpose though I would favour the K-S test, because binning the data takes away some information.
On the other hand maybe your hypothesis is that the distribution is normal and you want to find what the mean and dispersion are. In which case you can't use the K-S test, that's not what it is for. However you can minimise chi-squared to find the best fitting parameters using the binned data. A caveat here would be that when dealing with frequencies, chi-squared should not be used when you have small numbers per bin (say less than 9), because Poisson statistics become important. In these instances there are alternatives like the "Cash statistic".
I suppose at some level data are always binned. But when doing the K-S test there is usually only one object in each bin!
NB: The K-S test is not uniformly sensitive to differences in distributions at all values. It is most sensitive to differences in the median and very insensitive to differences in the tails of the distributions. A better test all round is the Anderson-Darling test is generally a better way of comparing two sample distributions.
