Re: Asymptotic Equipartition Property

From: Christian Kleinewaechter (christian.kleinewaechter_at_telefonica.de)
Date: 11/11/04 

Date: Thu, 11 Nov 2004 11:36:00 +0100

massimiliano1999@supereva.it wrote:
> Let Ae(n) to be the "typical set" with respect to p(x), where with typical
> set i mean the definition given in Information Theory.

You mean "one of the definitions given in information theory" ;-)

> I dont understand the following:
>
> My book defines the typical set Ae(n) with respect to p(x) to be the set
> of sequences (x_1,x_2,...x_n) with the proprerty that
> 1/(2^n(H(X)+e) <= p(x_1,x_2,...x_n) <= 1/(2^n(H(X)-e))
> where H(X) is the entropy of X, and X_1,X_2,...,X_n are identically
> distributed random variables according to p(x).
>
> The author proves that
> p(X_1,X_2,...,X_n) tends in probability to 2^(-nH(X)).
>
> So far i understood but i dont understand when the author says:
> "the probability of the event (X_1,X_2,...,X_n) in Ae(n) tends to 1 as
> n->+inf"
>
> For definition of event (x_1,x_2,...,x_n) in Ae(n),
> is 2^-n(H(X)+e)<= p(x_1,x_2,...,x_n)<=2^-n(H(X)-e))
> so when n tends to infinity both the left and the right values tend to 0,
> so p(x_1,x_n,...x_n) with n->inf tends to 0.
> Where are i wrong?
>

The number of sequences in Ae(n) tends to infinity for e fixed and n->inf.