Text 20, 101 rader
Skriven 2004-09-11 06:28:00 av Guy Hoelzer (1:278/230)
Ärende: Re: Dawkins gives incorre
=================================


in article chsg65$1hqg$1@darwin.ediacara.org, Tim Tyler at tim@tt1lock.org
wrote on 9/10/04 8:11 AM:

> Guy Hoelzer <hoelzer@unr.edu> wrote or quoted:
>> Tim Tyler at tim@tt1lock.org wrote on 9/9/04 8:25 AM:
>>> Guy Hoelzer <hoelzer@unr.edu> wrote or quoted:
>>>> in article chnea4$30dh$1@darwin.ediacara.org, Tim Tyler at tim@tt1lock.org
> 
>>>>> Shannon's information has proved to be a remarkably useful construct -
>>>>> AFAICS.
>>>> 
>>>> It has indeed.  Indeed, I often rely on it myself.  However, my use of the
>>>> Shannon/Weaver information measure (-p_i x log p_i,) and most usage in the
>>>> literature utterly ignores the role of perception because it has no
>>>> representation in the equation.  In fact, the equation is solely about the
>>>> degree of structure in the data.
>>> 
>>> p_i is the probability of symbol i arising at that point.
>> 
>> I have no evidence to offer regarding the actual frequency with which this
>> definition of p_i is used, but I do not use it to reference a "probability"
>> per se, and I am certain that many others use it as I do.  We treat it as a
>> frequency, which is objectively estimable from data.
> 
> In general it is not a frequency.  Probabilities are different from
> frequencies.

I know that.  This is why our two perspectives are different.

[snip]

>> In this way, the sensitivity of p_i to individual differences in
>> perception is minimized.
>> This transition in the meaning of p_i happened without much notice for many
>> (most? The vast majority?) information scientists because it increased the
>> extent of objectivity and measurement agreement among scientists, and
>> extended the utility of information theory throughout the physical sciences.
> 
> Nothing fitting this description ever happened.

Are you arguing that treating p_i as frequency is almost never done, or that
this practice has not increased in frequency?  Or are you just arguing that
you don't think it has become sufficiently common to call it a transition?
 
[snip]

>>> An observer who knows what symbol is coming next (because he
>>> has seen a very similar message before) will assign different
>>> probabilites to the symbols from an observer who is seeing
>>> the message for the first time - and both will assign different
>>> probabilities from an observer who is not even familiar with
>>> the language the message is written in.
>> 
>> This is a nice description of the (severe IMHO) limitations of the
>> "telegraph"-context-laden version of the theory that Shannon originally
>> devised for his telegraph-company employer.  With all of your protestation
>> about my lack of fidelity to Shannon's original context, you haven't
>> suggested any reasons why treating p_i as a frequency, rather than a
>> probability, is problematic.  Can you think of any such problems? [...]
> 
> The ones above?

I didn't see any problems suggested in your previous post or in the material
I snipped above, which was a description of how probabilities and
frequencies differ.  Your argument then seemed to consist merely of saying
that Shannon originally meant p_i to be a probability, rather than a
frequency, to which I already agreed.  None of that addresses my question.
 
> p_i can only be treated as a frequency, *if* the source is something like
> a RNG - where the probability of symbols being emitted is constant - and
> does not depend on the history of the stream or environmental conditions.

That may be the condition under which a probability and a frequency are
interchangeable, but it still does not address the issue at hand.  Given the
differences between probabilities and frequencies, why isn't it better to
think of p_i as a frequency instead of a probability as Shannon first had in
mind?
 
> That is certainly not the general case - and it is not the case with many
> common sorts of message streams either.
>  
>> If not, then don't you think it is worth considering the more extensive
>> version of the theory?
> 
> It isn't "more extensive".

It is more absolutely extensive in its potential application and the breadth
of its explanatory power because it overcomes the limitations of trying to
approximate identical states among observers before they can agree upon the
information content in a data set (an observation).

Guy
---
ū RIMEGate(tm)/RGXPost V1.14 at BBSWORLD * Info@bbsworld.com

---
 * RIMEGate(tm)V10.2á˙* RelayNet(tm) NNTP Gateway * MoonDog BBS
 * RgateImp.MoonDog.BBS at 9/11/04 6:28:23 AM
 * Origin: MoonDog BBS, Brooklyn,NY, 718 692-2498, 1:278/230 (1:278/230)