Asymptotic Equipartition Property

massimiliano1999_at_supereva.it
Date: 11/08/04 

Date: Mon, 08 Nov 2004 13:49:27 GMT

Let Ae(n) to be the "typical set" with respect to p(x), where with typical
set i mean the definition given in Information Theory.
I dont understand the following:

My book defines the typical set Ae(n) with respect to p(x) to be the set
of sequences (x_1,x_2,...x_n) with the proprerty that
1/(2^n(H(X)+e) <= p(x_1,x_2,...x_n) <= 1/(2^n(H(X)-e))
where H(X) is the entropy of X, and X_1,X_2,...,X_n are identically
distributed random variables according to p(x).

The author proves that
p(X_1,X_2,...,X_n) tends in probability to 2^(-nH(X)).

So far i understood but i dont understand when the author says:
"the probability of the event (X_1,X_2,...,X_n) in Ae(n) tends to 1 as
n->+inf"

For definition of event (x_1,x_2,...,x_n) in Ae(n),
is 2^-n(H(X)+e)<= p(x_1,x_2,...,x_n)<=2^-n(H(X)-e))
so when n tends to infinity both the left and the right values tend to 0,
so p(x_1,x_n,...x_n) with n->inf tends to 0.
Where are i wrong?

Relevant Pages

  • Typical set vs. smallest delta-sufficient subset
    ... contains almost all the probability. ... we have that the typical set and the smallest delta-sufficient ... But the typical set doesn't contain the most probable outcomes (as for ...
    (comp.compression)
  • Re: Asymptotic Equipartition Property
    ... > set i mean the definition given in Information Theory. ... > My book defines the typical set Aewith respect to pto be the set ... The number of sequences in Aetends to infinity for e fixed and n->inf. ...
    (comp.theory)