Quantification of uncertainty, or information loss, when modeling a physical system? Say I have a physical circuit (say a battery, wires and an incandescent lamp) that I want to model.
To model it I could perhaps use:


*

*traditional circuit theory 

*some kind of finite element analysis

*emtp
Whichever method I use I lose some information and introduce uncertainty (the arrangement of atoms in the wire, the exact room temperature etc). 
I want to compare the techniques based on this information loss. I'm thinking about it like this:
If the physical circuit was taken away and I had to recreate the physical circuit using my model (assume that I have a magic atom arrangement machine - so no limitation in rebuilding),  the resulting circuit would differ from the original by some maximum amount of information. This amount would be different for the three techniques given above and would act as a comparison metric for the modelling techniques.
How should I quantify this information loss?
Am I thinking about this in a sensible way?
 A: This actually extends beyond just computational approaches and applies to experimental approaches also. And it's not at all a trivial problem to address. Generally speaking, we construct a model of some physical system -- either computationally or experimentally -- and we make certain assumptions to simplify the problem. In your circuit example, maybe we neglect the resistance of the wires in our simulation. Or maybe I am designing an airliner and I make a 1/100 scale model to test in a water tunnel experimentally. In all cases, before we can quantify uncertainty, we need to describe our uncertainty:


*

*Uncertainty in our assumptions. We neglect resistance in the wires, or we assume things scale with Reynolds number in our small-scale experiments, or we assume that relativity doesn't matter. So start by listing your assumptions and what they mean and how they differ from real life.

*Uncertainty in our inputs. We simulate a circuit and specify the voltage of our battery. But how certain are we of that voltage? We say $g = -9.8$ or $\pi = 3.14$, but we know there are more digits than that and so that's another example of uncertainty in the input.

*Uncertainty in the measurements/methodology. Every experimental measuring system has uncertainty, every computational model has discretization errors, every CPU has different floating point behavior. List and understand those errors. 
Now that you have a list of all of your possible sources of uncertainty, you have to figure out how to quantify their impact. This is going to be very specific to your methods, but there are some general approaches. 
If you can assume some statistical information about your errors, for example your real-life battery has a voltage of $V = 9 \pm 0.01$ and you can assume the errors are normally distributed, you can perform a Monte-Carlo simulation of your simulation by drawing inputs from the distribution of inputs and measuring the resulting output. There are numerous platform to help do this, the most useful (in my field) is Dakota. 
If you have analytical functions for things, you can analytically perturb your inputs (Taylor series) and see how those errors propagate through your system.
If you have errors in your measurements (experimental or computational), you can derive confidence intervals and perform other statistical tests.
If you are running simulations, you would perform grid refinement studies. Of if that's too difficult, you can use Richardson Extrapolation by fixing your grid and changing the order of accuracy of your method and seeing how the results change. 
If you make a bunch of assumptions, perform perturbation analysis on the equations of your model and see if those assumptions are valid or not. 
Basically, UQ is a huge field and it covers both computational and physical domains. Depending on the cost of your model, you may not be able to do it. If it costs 1 million CPU hours to run a simulation and that simulation has 200 inputs, it's unlikely that you will be able to run a Monte Carlo. It just won't happen. So you'll have to dig deeper into your domain and see what is possible. 
But you'll want to start by making sure you are aware of all of your assumptions, your inputs, your techniques, and your expected outputs, otherwise you'll never be able to define or quantify uncertainties. 
