15-08-2013, 10:19 PM
Albert Rossi Wrote:Albert Rossi Wrote:Charles Drago Wrote:Albert,
Would I be correct, then, if I rewrote "information content is inversely proportional to probability" thusly:
"Information content is inversely proportional to randomness."
No, actually it is the opposite. The greater the randomness, the greater the entropy, which is a measure of information.
The problem here is that Shannon and Information Theory use the concept of information in a rather counter-intuitive way. Common usage of the term would suggest that random = meaningless. But "information" is not the same as "meaning" in the context of this theory. I sometimes fall into the same trap and have to correct myself. Information theory was invented to model systems of communication, and is not a theory of semantics.
Another way of thinking about that difference is the idea of "structure" vs "information". Semantic content seems to be closely associated with structure, but structure is somewhat different from the notion of information content in Shannon's sense. A really interesting read, if you are inclined to read such things, which deals with this interaction, and with the notion of "complexity" (with a special view to the problem of the evolution of complexity from very simple processes) is by the Nobel Prize winning physicist:
Murray Gell-Mann, The Quark and the Jaguar: Adventures in the Simple and the Complex. (see http://www.amazon.com/The-Quark-Jaguar-A...0805072535 -- not a suggestion to buy it there, though).
Thanks much, Albert. I think that I get it.
I'm especially grateful for the recommendation, of which I'll take full advantage right now. Bezos, here I come!
Charles Drago
Co-Founder, Deep Politics Forum
If an individual, through either his own volition or events over which he had no control, found himself taking up residence in a country undefined by flags or physical borders, he could be assured of one immediate and abiding consequence: He was on his own, and solitude and loneliness would probably be his companions unto the grave.
-- James Lee Burke, Rain Gods
You can't blame the innocent, they are always guiltless. All you can do is control them or eliminate them. Innocence is a kind of insanity.
-- Graham Greene
Co-Founder, Deep Politics Forum
If an individual, through either his own volition or events over which he had no control, found himself taking up residence in a country undefined by flags or physical borders, he could be assured of one immediate and abiding consequence: He was on his own, and solitude and loneliness would probably be his companions unto the grave.
-- James Lee Burke, Rain Gods
You can't blame the innocent, they are always guiltless. All you can do is control them or eliminate them. Innocence is a kind of insanity.
-- Graham Greene