According to the Wikipedia,

""Information of an entity is that which resolves the uncertainty of its properties""

source: Wikipedia, physical information

Information contained in a physical system = the number of yes/no questions you need to get answered to fully specify the system.

But there are more then one perspective about information notion, p.e.

Shannon information theory, usually called just ‘information’ theory and Kolmogorov complexity theory the ‘algorithmic information’ theory,

Both theories aim at providing a means for measuring‘information’.

Both are measures of the smallest number of bits that can be used to regenerate a string perfectly.

With the signifiant difference that the Kolmogorov complexity approach is in an algorithm context

Shannon entropy can be considered in approximation as the estimation of the Kolmogorov complexity.

The concept of Entropy is the common thread that connects physical world (mass, energy) the world of uncertainity of events or existent (Information).

The informationn concept seems to be quite complex indeed already a question can emerges about the true ""nature"" of information,... and the link made by Entropy between Energy, Mass and Information

see this post :

"Is there an equivalence between information, energy and matter?""

but come back to Shannon Entropy and its measure:

Shannon Entropy of well-encrypted stream is equal to Shannon Entropy of random noise.

...information can not be define only based on data...? but we can ask the question if in the Kolmogorov complexity context, the algorithm used can be considered as a ""knowledge"" about ""reciever""..?

Are knowledge about data and receiver are needed,....?

Change in the receiver's state would be the only way to measure how much information is contained in the message.. ? ....so subjective

how can go deeper in the problematic about the true ""nature"" of information ..?

  • $\begingroup$ Consider to include links of quotes. $\endgroup$ – Qmechanic Apr 20 at 6:44
  • $\begingroup$ @JohnForkosh following the link you provide from wikipedia: "In algorithmic information theory (a subfield of computer science and mathematics), the Kolmogorov complexity of an object, such as a piece of text, is the length of the shortest computer program (in a predetermined programming language)"... it seems we know something about the ""predetermined programming language use by the reciever... $\endgroup$ – alamata Apr 20 at 7:34
  • $\begingroup$ @JohnForkosh..indeed i m really interressed by the notion of information..Thx for the link provided about Kolmogorov Complexity....But there are more then one perspective about information notion, per example, the ‘information’ theory and the ‘algorithmic information’ theory, both theories aim at providing a means for measuring‘information’..... $\endgroup$ – alamata Apr 20 at 18:38

change in the receiver's state is the only way to measure how much information is contained in the message..so subjective

Well, yes, it can be subjective. In the case of natural language, where a variety of nuances and redundancies exist, one way to measure information is to drop a word, then ask native speakers to fill it in.

If no one can guess the word, it has high information content. If everyone can guess it, it has little, or none. Poetry of Emily Dickinson has high information content per word, and warranty terms and conditions... score very low.

For engineered systems, though, it is possible to make simple calculations of error rates (and apply redundancy to make those rates virtually zero), because there ARE completely known symbol systems. Natural language, however, is hard to formulate according to simple rules, because style, vocabulary, and dialect all are as complex as a human mind. There is no reason to avoid subjectivity, after all, in communication; that's very much the essence of Emily's works.

It is important to note that information is not the same as meaning; some information is important to us, while other information, adequately transmitted, will be disregarded. The total information of this text, might include the type font and line-break points, but our communication channel can change those (lose that information) without anyone taking notice.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.