PDF4PRO ⚡AMP

Modern search engine that looking for books and documents around the web

Example: biology

Entropy and Mutual Information

Entropy and Mutual Information Erik G. Learned-Miller Department of Computer Science University of Massachusetts, Amherst Amherst, MA 01003. September 16, 2013. Abstract This document is an introduction to Entropy and Mutual Information for discrete random variables. It gives their definitions in terms of prob- abilities, and a few simple examples. 1. 1 Entropy The Entropy of a random variable is a function which attempts to characterize the unpredictability of a random variable. Consider a random variable X rep- resenting the number that comes up on a roulette wheel and a random variable Y representing the number that comes up on a fair 6-sided die. The Entropy of X is greater than the Entropy of Y . In addition to the numbers 1 through 6, the values on the roulette wheel can take on the values 7 through 36. In some sense, it is less predictable. But Entropy is not just about the number of possible outcomes. It is also about their frequency.

represents whether the roll is even (0 if even, 1 if odd). Clearly, the value of Y tells us something about the value of X and vice versa. That is, these variables share mutual information. On the other hand, if X represents the roll of one fair die, and Z represents the roll of another fair die, then X and Z share no mutual information. The roll

Tags:

  Even

Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Spam in document Broken preview Other abuse

Transcription of Entropy and Mutual Information

Related search queries