Text 67, 125 rader
Skriven 2004-09-15 21:58:00 av Guy Hoelzer (1:278/230)
Ärende: Re: Dawkins gives incorre
=================================


in article ci7mqk$24qd$1@darwin.ediacara.org, Tim Tyler at tim@tt1lock.org
wrote on 9/14/04 2:11 PM:

> Guy Hoelzer <hoelzer@unr.edu> wrote or quoted:
>> in article chvng2$2hqs$1@darwin.ediacara.org, Tim Tyler at tim@tt1lock.org:
>>> Guy Hoelzer <hoelzer@unr.edu> wrote or quoted:
>>>> in article chsg65$1hqg$1@darwin.ediacara.org, Tim Tyler at
tim@tt1lock.org:
>>>>> Guy Hoelzer <hoelzer@unr.edu> wrote or quoted:
> 
>>>> Are you arguing that treating p_i as frequency is almost never done,
>>>> or that this practice has not increased in frequency?  Or are you
>>>> just arguing that you don't think it has become sufficiently common
>>>> to call it a transition?
>>> 
>>> p_i is /always/ the probability of the i'th symbol arising.
>>> 
>>> Sometimes the probabilities are determined completly by symbol frequencies
>>> - but the p_i's are never frequencies.
>> 
>> If they are "determined completely by by symbol frequencies" then they are
>> frequencies.
> 
> A frequency is normally a measurement of the number of times that a
> repeated event occurs per unit time.

I am aware of that definition, but I am using a different conventional
meaning.  This distinction might be a source of some of our differences.
The definition I am using is the one I believe to be most commonly used in
the biological sciences, and it well represented by the one expressed by "A
Dictionary of Ecology, Evolution, and Systematics."  It reads:

"The number of items belonging to a category or class; the number of
occasions that a given species occurs in a series of examples."

This dictionary does not list any other definitions for "frequency."
 
>> I must say I am quite surprised at your continuing insistence that model of
>> information is unlike anything in the minds of scientists publishing in this
>> area.  Are you unaware of the current debate over the meanings of both
>> entropy and information in the context of order/disorder (dispersion)?
> 
> That question seems too vague to answer.  People debate all kinds of
> stuff all the time.  AFAIK, the meanings of the terms "entropy" and
> "information" are not especially controversial at the moment.

I do see them as " especially controversial at the moment," but there is no
reason we can't disagree on this.
 
>> How do you explain the information theoretical methods of analysis,
>> such as the Akaike Information Content measure, that have been growing
>> fast in application.  It is fundamental to these methods that they
>> yield precisely the same result in the hands of every scientist, so
>> that they are repeatable and verifiable. The role of perceiver, which
>> was Shannon's initial concern, has been dropped from information theory
>> by many.
> 
> I'm not sure about the Akaike Information Criterion, but - as far as
> I can tell - is escapes observer-dependence by completely specifying
> a particular hypothetical observer (its model) and then asking how
> effective that observer is at predicting the data.
> 
> In other words, the term "information" in its title appears to refer
> not to the information gained by someone measuring its value - but to
> the information that can be expected to be gained by a completely-
> specified hypothetical observer witnessing the data stream.

A good resource for learning about AIC and its application (IMHO) is the
book:

Burnham, K. P., and D. R. Anderson. 1998. Model selection and inference: a
practical information-theoretic approach. Springer-Verlag, New York, New
York, USA. 353 pp.

The authors explain why Kullback-Leibler information is more fundamental
than Shannon information and show that it is more general (it includes
Shannon information). It is Kullback-Leibler that is assumed under the AIC
paradigm, which does not posit an hypothetical observer, according to the
authors.  Instead, they argue, the set of AIC values (or adjusted analogues,
such as AICc) that you get out of a comparative analysis express the
relative distance of competing models from objective Truth.  That claim took
me by surprise when I first ran across it, but you really have to examine
the theory closely to make an informed judgment about it.
 
>>> They always add up to 1.0 - like probabilities do.
>> 
>> Like frequencies always do.
> 
> Frequencies are usually measured in Hertz - and never add up to a
> dimensionless quantity such as 1.0.
> 
> Indeed, adding the values of frequencies together is usually a bad move:
> since 1hz+2hz != 3hz.

Under the definition provided above frequencies must always add to one if
you have included all possible types in your data.  For example, if you
consider the frequency of each allele present in a data set, those
frequencies must add to one.
 
> Have you encountered Chaitin's algorithmic information theory?

Sure.  I have even discussed it with him for a few minutes.
 
> It doesn't appear to be what you are talking about - but it shares
> the element of observer-independence (though it tends to become
> language-dependent in the process).

You are correct that this is not exactly what I am talking about, but I do
not see how it is observer-dependent.  It is clear to me why it is
language-dependent, but it seems to me that different observers using the
same language would always calculate the same value of Chaitin algorithmic
information given the same data.

Cheers,

Guy
---
ū RIMEGate(tm)/RGXPost V1.14 at BBSWORLD * Info@bbsworld.com

---
 * RIMEGate(tm)V10.2á˙* RelayNet(tm) NNTP Gateway * MoonDog BBS
 * RgateImp.MoonDog.BBS at 9/15/04 9:58:06 PM
 * Origin: MoonDog BBS, Brooklyn,NY, 718 692-2498, 1:278/230 (1:278/230)