INTRODUCTION TO INFORMATION THEORY

1 INTRODUCTION TO INFORMATION THEORY {ch:intro_info}This chapter introduces some of the basic concepts of INFORMATION THEORY , as wellas the definitions and notations of probabilities that will be used throughoutthe book. The notion of entropy, which is fundamental to the whole topic ofthis book, is introduced here. We also present the main questions of informationtheory, data compression and error correction, and state Shannon s Random variablesThe main object of this book will be the behavior of large sets ofdiscreterandom variables. A discrete random variableXis completely defined1bythe set of values it can take,X, which we assume to be a finite set, and itsprobability distribution{pX(x)}x X. The valuepX(x) is the probability thatthe random variableXtakes the valuex. The probability distributionpX:X [0,1] must satisfy the normalization conditionXx XpX(x) = 1.

INTRODUCTION TO INFORMATION THEORY {ch:intro_info} This chapter introduces some of the basic concepts of information theory, as well as the definitions and notations of probabilities that will be used throughout the book. The notion of entropy, which is fundamental to the whole topic of this book, is introduced here.

Loading..

Tags:

  Introduction

Information

Related search queries