I'm planning an experiment, but due to cost and other circumstances, there will only be one realization of it. So the plan is to collect as many different types of data as we can with a variety of techniques and sift through it. However, I'm not sure how to address quantifying or estimating the uncertainty in any given set of final results.
I can run any number of tests on subsets or components of the experiment and the data collection using representative conditions -- maybe scaled down, or maybe isolating certain aspects, etc..
If we assume everything is deterministic, how can I take uncertainty from these proxy/surrogate experiments using components to estimate the uncertainty (or to establish a confidence interval) on the final data from the full experiment?
As an example, let's say that my experiment is to run a large reacting flow experiment. I will have many different cameras aimed at it measuring different things (via various filters, lasers, and so on) and I will have many different pressure sensors and temperature sensors. I cannot run the full experiment more than once, but I can test any of the measurement systems on smaller, more representative problems:
- Cameras can be set up to look at simple flames, like a Bunsen burner;
- Pressure sensors can be hooked up to tubes with speakers;
- Temperature sensors can be placed in various heated flows.
For any of those simple setups, I can vary anything I want and run as many times as I want. So, I could run my camera with a large range of possible settings and possible optical paths and get the uncertainty in a given measurement due to those changes in settings. I could run the pressure sensors over a range of frequencies, amplitudes, sampling rates, etc. and measure the uncertainty there.
But, how can those uncertainties measured from simpler experiments inform the confidence in the full experiment? Because the full system is very complicated, there is no complete set of model equations to propagate uncertainty through. Since I cannot run more than once, I cannot vary settings and evaluate sensitivities. I can only run proxy experiments and have to somehow use that information to estimate a confidence or uncertainty in the full experiment.