## Rate Distortion Theory February 20, 2006

*Source coding* aims to reduce redundancy in information to improve storage/transmission efficiency, while *channel coding* aims to increase redundancy to improve reliable transmission over a noisy channel. The source coding theorem:

Given a discrete memoryless source of entropy,

H(ζ), the average code-word lengthLfor any distortionless source encoding scheme, is bounded asL ≥ H(ζ)

In a system, *source symbols* with alphabet *X~p(X=x _{i})* go through an encoder and become

*representation symbols*with alphabet

*Y~p(Y=y*

_{j})The single letter distortion measure, *d(x _{i}, y_{j})* is defined as the cost in representing

*x*by

_{i}*y*. Define

_{j}*p(y _{j}| x_{i})* is said to be “

*D*-admissible” iff

*d≤D*for some

*D*. The set of

*D*-admissible

*p(y*is denoted by

_{j}| x_{i})*P*

_{D}= {p(y_{j}| x_{i}) : d ≤ D}**[Mutual Information]**

The mutual information, *I(X; Y)* is given by

**[Rate Distortion Measure]**

A rate distortion function, *R(D)* (and measured in *bits*) is defined as the smallest coding rate possible for which the average distortion, *d* is guaranteed not to exceed D.

subject to the constraint

## Leave a Reply

You must be logged in to post a comment.