Transcription of Chapter 2: Entropy and Mutual Information - UIC
{{id}} {{{paragraph}}}
Example: dental hygienist
Chapter 2: Entropy and Mutual Information - UIC
University of Illinois at Chicago ECE 534, Fall 2009, Natasha DevroyeUniversity of Illinois at Chicago ECE 534, Fall 2009, Natasha DevroyeChapter 2: Entropy and Mutual InformationUniversity of Illinois at Chicago ECE 534, Fall 2009, Natasha DevroyeChapter 2 outline Definitions Entropy Joint Entropy , conditional Entropy Relative Entropy , Mutual Information Chain rules Jensen s inequality Log-sum inequality Data processing inequality Fano s inequalityUniversity of Illinois at Chicago ECE 534, Fall 2009, Natasha DevroyeDefinitionsA discrete random variableXtakes on valuesxfrom the discrete probability mass function (pmf) is described bypX(x)=p(x) = Pr{X=x},forx of Illinois at Chicago ECE 534, Fall 2009, Natasha DevroyeDefinitionsCopyright Cambridge University Press 2003.
University of Illinois at Chicago ECE 534, Fall 2009, Natasha Devroye Chapter 2: Entropy and Mutual Information University of Illinois at Chicago ECE 534, Fall 2009, Natasha Devroye
Loading..
Tags:
Information
Domain:
Source:
Link to this page:
Please notify us if you found a problem with this document: