I have been reading the chapter "Maxwell's Demon and Shannon's choice" on N. Katherine Hayles' book "Chaos Bound: Orderly Disorder in Contemporary Literature and Science" where she compares different definitions of information: Shannon information which is equivalent to Shannon entropy vs. thermodynamic information where she references to Léon Brillouin's negentropy concept as information. She adds that although they have completely opposite signs in their definitions and although the probabilities they are talking about are completely different concepts (one based on ignorance of microstates, the other about not knowing about the message chosen), what they see is two different points of view looking at the same thing ("asking whether Shannon or Brillouin made a better definition of information is similar to asking if a glass is half empty or half full") Their difference is that "Shannon considers the uncertainty in the message at its source, whereas Brillouin considers it at the destination". She then points out to the binary entropy function:

Binary entropy function

And adds this visualization about the difference between the two points of view:

Brillouin and Shannon

I can follow up to the point where Shannon relates to this graph, but cannot follow how thermodynamic entropy and Brillouin's "negentropy as information" concept can be tied to this diagram. Since this not a science book, I am not really sure if any of it is even valid. So, any hint would be appreciated...


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.