Median filtering - cheating? The concept is simple, and is used to remove non-typical data. A sample set is put in numerical order and the extremes discarded. For example, in an experiment I am doing I take batches of 10 measurements and discard all but the center 4 results. It improves the reliability (relative standard deviation) of the final results enormously.
How legitimate is this in general in physics? I can certainly imagine situations where doing this by hand would be considered to be falsifying the data.
 A: When you say "improves the reliability", well that is not clear at all, because you have reduced your sample size and possibly introduced an (unknown) bias.
Median filtering is typically used where you do not fully understand the noise properties of your sample and where there may be cases of results that are way out from the expected result because of rare phenomena that cannot be fully characterised. i.e Distinctly non-Gaussian tails in the distribution that might enable you to characterise the distribution in terms of something that is more well behaved plus "outliers".
So an example from astronomy might be the averaging of a number of spectra of the same object, where an occasional pixel in a spectrum may be contaminated by a cosmic ray strike. Such events will lead to a small, but distinctly non-Gaussian tail in the distribution of measured values. Taking the mean in such circumstances, would give crazy values at the positions of pixels affected by the cosmic rays in one of the spectra, so the median is preferred.
The experiment you describe does not sound like that, unless you have a plausible reason why more than half your data are no good. Yes it will extract a signal, but whether that signal could be considered "reliable" is moot in those circumstances. Certainly the standard deviation of the remaining objects would be no indicator of the noise. A better noise estimate would be the median absolute deviation of the whole sample.
