ECE 531: Detection and Estimation Theory

1 ece 531 : Detection and Estimation Theory Natasha Devroye ~devroye Spring 2011. Example of Detection Example of Estimation Goals infer value of unknown state of nature based on noisy observations Mathematically, optimally Noise Transmission /. Nature Processing measurement phenomenon model of observation decision rule experiment or transmission Estimation function process the ``truth''. mapping from observation space model of hypothesis H to decisions/. estimates Detection example 1: digital communications Noise Source Encoder Channel Decoder Destination 10001010100010. Detect? Detection example 2: Radar communication Send Receive Hypothesis Detect?

2 Hypothesis Further examples Sonar: enemy submarine Image processing: detect an aircraft from infrared images Biomedicine: cardiac arryhthmia from heartbeat sound wave Control: detect occurrence of abrupt change in system to be controlled Seismology: detect presence of oil deposit Difference between Detection and Estimation ? Detection : Discrete set of hypotheses Right or wrong Estimation : Continuos set of hypotheses Almost always wrong - minimize error instead Estimation example 1: communications Pulse amplitude modulation (PAM). Analog source Sampler Transmitter Receiver? Estimation example 2: Radar Send Receive Hypothesis Estimate?

3 Hypothesis Our methods Will treat everything generally, with a unified mathematical representation Bias towards Gaussian noise and linear observation - parameter models Examples mainly drawn from communications / radar Aside: Classical vs. Bayesian . Classical Hypotheses/parameters are fixed, non-random Bayesian Hypotheses/parameters are treated as random variables with assumed priors (or a priori distributions). Other useful references: Course Harry Textbook: L. Van Fundamentals Trees, Detection , of Statistical Estimation , Signal Processing, and Modulation Theory , PartVolume I, II, 1: III, Estimation IV Theory , by , Vincent Kay, Introduction Prentice Hall, to 1993.

4 Signaland (possibly). Detection andFundamentals Estimation of Statistical Signal Processing, Volume Louis 2: Detection L. Scharf Theory , and Cedric by Steven Demeure, M. Kay, Statistical Prentice Signal Hall 1998. Processing: Detection , Estimation , and Time OtherAnalysis Series useful references: Carl Helstrom, Harry Elements L. Van Trees, of SignalEstimation, Detection , Detection and Estimation . ModulationIt's out ofPart Theory , print, so here's I, II, III, IVmy pdf copy. H. Vincent Poor, Introduction to Signal Detection and Estimation Course outline Notes: Louis main Series follow Scharf points andthe courseDemeure, Cedric textbooksStatistical fairly closely, using Signal a mixtureDetection, Processing: of slides (highlighting Estimation , and and with nice illustrations) and more in-depth blackboard derivations/proofs in class.

5 I. Analysis the Time will Carlpost a pdf version Helstrom, of the Elements of slides SignalasDetection they becomeand ready here, but Estimation . It'sthe outderivations of print, sowill be my here's givenpdfincopy. class only. Notes: Fundamentals I will follow of Statistical theProcessing, Signal course textbooks Volume 1:fairly closely, Estimation usingbya Steven Theory , ,slides (highlighting Prentice Hall, 1993the Topics: Estimation main points Theory : and with nice illustrations) and more in-depth blackboard derivations/proofs in class. I. General will postMinimum Variance a pdf version Unbiased of the slides asEstimation, they become , 5 here, but the derivations will be given in ready Cramer-Rao class only.

6 Lower Bound, Linear Models+Unbiased Estimators, , 6. Maximum LikelihoodTheory: Topics: Estimation Estimation , Least squares Estimation , General Minimum Variance Estimation , , 5. Bayesian Estimation , Cramer-Rao Lower Bound, Kalman Detection filtering Estimators, , 6. Theory : Linear Models+Unbiased Statistical Maximum Detection LikelihoodTheory, Estimation , Deterministic Fundamentals Signals, of Statistical Signal Processing, Volume 2: Detection Theory , by Steven M. Kay, Prentice Hall 1998. Least squares Estimation , Random Signals, Bayesian Estimation , Statistical Detection Theory 2, Detection Theory : Non-parametric and robust Detection Statistical Detection Theory , Deterministic Signals, Grading: Weekly homeworks (15%), Exam 1 = max(Exam1, Exam 2, Final) (20%), Exam 2 =.

7 Random Signals, max(Exam 2, Final) (20%), Project (15%), Final exam (30%). Statistical Detection Theory 2, Non-parametric and robust Detection Grading: Weekly homeworks (15%), Exam 1 = max(Exam1, Exam 2, Final) (20%), Exam 2 =. 1 of 3 1/11/10 8:50 PM. max(Exam 2, Final) (20%), Project (15%), Final exam (30%). 1 of 3 1/11/10 8:50 PM. Estimation : General Minimum Variance Unbiased Estimation Bias: (expected value of estimator - true value of data). MVUE: Estimation : Cramer-Rao lower bound Lower bound on variance of ANY unbiased estimator! Usage: assert whether an estimator is MVUE. benchmark against which to measure the performance of an unbiased estimator feasibility studies Noise Depends on?

8 Transmission /. Nature Processing measurement Estimation : linear models What's a linear model and why is it useful? What can be said? Best Linear Unbiased Estimators (BLUE). Estimation : Maximum Likelihood Estimation Alternative to MVUE which is hard to find in general Easy to compute - very widely used and practical What is the MLE? Properties? Estimation : Least Squares Alternative estimator with no general optimality properties, but nice and intuitive and no probabilistic assumptions on data are made - only need a signal model Advantages? Disadvantages? Estimation : Bayesian Estimation Parameter to be estimated is assumed to be random, according to some prior distribution which models our knowledge of it Bayesian Minimum Mean Squared Error (MMSE): Applications to Gaussian noise / linear model Estimation : Bayesian Estimation General risk functions - arbitrary cost functions Maximum a posteriori (MAP) Estimation Linear MMSE: constrain estimator to be linear - very practical Estimation : Kalman filtering recursive filter for estimating internal state of linear dynamical system from a series of noisy measurements example.

9 Tracking a moving target using radar measurements noisy measurements linear dynamical system estimate the position and/or velocity Estimation : Kalman filtering recursive filter for estimating internal state of linear dynamical system from a series of noisy measurements than can recursively estimate/predict and update the state covariances as: Other useful references: Course Harry Textbook: L. Van Fundamentals Trees, Detection , of Statistical Estimation , Signal Processing, and Modulation Theory , PartVolume I, II, 1: III, Estimation IV Theory , by , Vincent Kay, Introduction Prentice Hall, to 1993. Signaland (possibly).

10 Detection andFundamentals Estimation of Statistical Signal Processing, Volume Louis 2: Detection L. Scharf Theory , and Cedric by Steven Demeure, M. Kay, Statistical Prentice Signal Hall 1998. Processing: Detection , Estimation , and Time OtherAnalysis Series useful references: Carl Helstrom, Harry Elements L. Van Trees, of SignalEstimation, Detection , Detection and Estimation . ModulationIt's out ofPart Theory , print, so here's I, II, III, IVmy pdf copy. H. Vincent Poor, Introduction to Signal Detection and Estimation Course outline Notes: Louis main Series follow Scharf points andthe courseDemeure, Cedric textbooksStatistical fairly closely, using Signal a mixtureDetection, Processing: of slides (highlighting Estimation , and and with nice illustrations) and more in-depth blackboard derivations/proofs in class.