In my data analysis course we were taught that whenever we perform a measurement, the measured value is to be interpreted as the mean of the distribution for the true value. And the error in measurement is to be interpreted as the standard deviation of the distribution. The full distribution may be taken as Poisson or Exponential of Gaussian depending on the kind of experiment being performed. In general, the Gaussian distribution is preferred because of the Central Limit Theorem.
The book says:
For the experimenter the true value is an unknown parameter of the distribution. The measurement and its error are estimates of the true value and of the standard deviation of the distribution. (Remark that we do not need to know the full error distribution but only its standard deviation.)
-"Introduction to Statistics and Data Analysis for Physicists" by Bohm and Zech Pg. 83
But I have always been very uncomfortable with this interpretation as it does not bound the true value to any interval and allows it to take any value although with diminishing probability. I can think of atleast one context in which this interpretation seems absurd:
Suppose I wish to measure the length of a rod using a metre scale and the marking of, say, 1.414m seems to coincide with the length of the rod. Then, although I cannot conclude that the length is precisely 1.414m because of an error involved due to the limited least count of our measuring instrument which is impossible to get around as lengths are "continuous". I still do know precisely, ignoring systematic errors, that the length is no more than 1.415m and no less than 1.413m.
If I employ the conventional interpretation to this experiment, the true value is supposed to be a number lying anywhere in the distribution but with probability decreasing as the value goes more and more standard deviations away from the mean. But that is absurd since it cannot lie outside $(1.413, 1.415)$.
So, should there be a need to devise a new method of expressing measurements using a range of real numbers within "hard" limits rather than a mean and a standard deviation?