DART Tutorial Part IV - Mesoscale Research Group

1 DART Tutorial Part IV: Other Updates for an Observed Variable UCAR The Na'onal Center for Atmospheric Research is sponsored by the Na'onal Science Founda'on. Any opinions, ndings and conclusions or recommenda'ons expressed in this publica'on are those of the author(s) and do not necessarily re ect the views of the Na'onal Science Founda'on. Product of Two Gaussians p(B | AC)p(A | C) p(B | AC)p(A | C). p ( A | BC ) = =. p(B | C) p(B | x)p(x | C)dx Ensemble lters: Prior is available as nite sample. Probability Prior Ensemble 0. 4 2 0 2 4. Don't know much about proper'es of this sample. May naively assume it is random draw from truth'. Product of Two Gaussians p(B | AC)p(A | C) p(B | AC)p(A | C). p ( A | BC ) = =. p(B | C) p(B | x)p(x | C)dx How can we take product of sample with con'nuous likelihood?

2 Probability Prior PDF. Prior Ensemble 0. 4 2 0 2 4. Fit a con'nuous (Gaussian for now) distribu'on to sample. Product of Two Gaussians p(B | AC)p(A | C) p(B | AC)p(A | C). p ( A | BC ) = =. p(B | C) p(B | x)p(x | C)dx Observa'on likelihood usually con'nuous (nearly always Gaussian). Obs. Likelihood Probability Prior PDF. Prior Ensemble 0. 4 2 0 2 4. If Obs. likelihood isn't Gaussian, can generalize methods below. Product of Two Gaussians p(B | AC)p(A | C) p(B | AC)p(A | C). p ( A | BC ) = =. p(B | C) p(B | x)p(x | C)dx Product of prior Gaussian t and Obs. likelihood is Gaussian. Posterior PDF. Obs. Likelihood Probability Prior PDF. Prior Ensemble 0. 4 2 0 2 4. Compu'ng con'nuous posterior is simple. BUT, need to have a SAMPLE of this PDF.

3 Sampling Posterior PDF: There are many ways to do this. Posterior PDF. Probability 0. 2 1 0 1 2 3. Exact proper'es of di erent methods may be unclear. Trial and error s'll best way to see how they perform. Will interact with proper'es of predic'on models, etc. Sampling Posterior PDF: Just draw a random sample ( lter_kind=5 in &assim_tools_nml). Posterior PDF. Probability Random Sample 0. 2 1 0 1 2 3. Sampling Posterior PDF: Just draw a random sample ( lter_kind=5 in &assim_tools_nml). Posterior PDF. Probability Random Sample; Exact Mean 0. 2 1 0 1 2 3. Can play games' with this sample to improve (modify) its proper'es. Example: Adjust the mean of the sample to be exact. Can also adjust the variance to be exact. Sampling Posterior PDF: Just draw a random sample ( lter_kind=5 in &assim_tools_nml).

4 Posterior PDF. Probability Random Sample; Exact Mean and Var. 0. 2 1 0 1 2 3. Can play games' with this sample to improve (modify) its proper'es. Example: Adjust the mean of the sample to be exact. Can also adjust the variance to be exact. Sampling Posterior PDF: Just draw a random sample ( lter_kind=5 in &assim_tools_nml). Posterior PDF. Probability Random Sample; Exact Mean and Var. 0. 2 1 0 1 2 3. Might also want to eliminate rare extreme outliers. NOTE: Proper'es of these adjusted samples can be quite di erent. How these proper'es interact with the rest of the assimila'on is an open ques'on. Sampling Posterior PDF: Construct a determinis'c' sample with certain features. Posterior PDF. Probability 0. 3 2 1 0 1 2 3 4.

5 For instance: Sample could have exact mean and variance. This is insu cient to constrain ensemble, need other constraints. Sampling Posterior PDF: Construct a determinis'c' sample with certain features ( lter_kind=6 in &assim_tools_nml; manually adjust kurtosis). Posterior PDF. Probability Kurtosis 3. 0. 3 2 1 0 1 2 3 4. Example: Exact sample mean and variance. Sample kurtosis (related to the sharpness/tailedness of a distribution). is 3, which is the expected value for a normal distribution. Start by assuming a uniformly-spaced sample and adjus'ng quadra'cally. Sampling Posterior PDF: Construct a determinis'c' sample with certain features ( lter_kind=6 in &assim_tools_nml; manually adjust kurtosis). Posterior PDF. Probability Kurtosis 2.

6 Kurtosis 3. 0. 3 2 1 0 1 2 3 4. Example: Exact sample mean and variance. Sample kurtosis 2: less extreme outliers, less dense near mean. Avoiding outliers might be nice in certain applica'ons. Sampling heavily near mean might be nice. Sampling Posterior PDF: First two methods depend only on mean and variance of prior sample. Probability Prior Ensemble 0. 4 2 0 2 4. Example: Suppose prior sample is (signi cantly) bimodal? Might want to retain addi'onal informa'on from prior. Recall that Ensemble Adjustment Filter tried to do this (Sec'on 1). Sampling Posterior PDF: First two methods depend only on mean and variance of prior sample. Posterior PDF. Probability Obs. Likelihood Prior PDF. Random Posterior Ensemble Prior Ensemble 0.

7 4 2 0 2 4. Example: Suppose prior sample is (signi cantly) bimodal? Might want to retain addi'onal informa'on from prior. Recall that Ensemble Adjustment Filter tried to do this (Sec'on 1). Ensemble Filter Algorithms: Ensemble Kalman Filter (EnKF) ( lter_kind=2 in &assim_tools_nml). Probability Prior Ensemble 0. 4 2 0 2 4. Classical' Monte Carlo algorithm for data assimila'on Ensemble Filter Algorithms: Ensemble Kalman Filter (EnKF) ( lter_kind=2 in &assim_tools_nml). Probability Prior Ensemble 0. 4 2 0 2 4. Again, t a Gaussian to the sample. Ensemble Filter Algorithms: Ensemble Kalman Filter (EnKF) ( lter_kind=2 in &assim_tools_nml). Probability Obs. Likelihood Prior Ensemble 0. 4 2 0 2 4. Again, t a Gaussian to the sample.

8 Are there ways to do this without compu'ng prior sample stats? Ensemble Filter Algorithms: Ensemble Kalman Filter (EnKF) ( lter_kind=2 in &assim_tools_nml). Probability Obs. Likelihood Random Draws from Obs. Prior Ensemble 0. 4 2 0 2 4. Generate a random draw from the observa'on likelihood. Associate it with the rst sample of the prior ensemble. Ensemble Filter Algorithms: Ensemble Kalman Filter (EnKF) ( lter_kind=2 in &assim_tools_nml). Probability Obs. Likelihood Random Draws from Obs. Prior Ensemble 0. 4 2 0 2 4. Have sample of joint prior distribu'on for observa'on and prior MEAN. Adjus'ng the mean of obs. sample to be exact improves performance. Adjus'ng the variance may further improve performance. Outliers are a poten'al problem, but can be removed.

9 Ensemble Filter Algorithms: Ensemble Kalman Filter (EnKF) ( lter_kind=2 in &assim_tools_nml). Probability 0. 4 2 0 2 4. For each prior mean/obs. pair, nd mean of posterior PDF. DART Tutorial Sec'on 6: Slide 24 Ensemble Filter Algorithms: Ensemble Kalman Filter (EnKF) ( lter_kind=2 in &assim_tools_nml). Probability 0. 4 2 0 2 4. Prior sample standard devia'on s'll measures uncertainty of prior mean es'mate. Ensemble Filter Algorithms: Ensemble Kalman Filter (EnKF) ( lter_kind=2 in &assim_tools_nml). Probability 0. 4 2 0 2 4. Prior sample standard devia'on s'll measures uncertainty of prior mean es'mate. Obs. likelihood standard devia'on measures uncertainty of obs. es'mate. Ensemble Filter Algorithms: Ensemble Kalman Filter (EnKF) ( lter_kind=2 in &assim_tools_nml).

10 Posterior PDF. Probability 0. 4 2 0 2 4. Take product of the prior/obs distribu'ons for rst sample. This is the standard Gaussian product. Ensemble Filter Algorithms: Ensemble Kalman Filter (EnKF) ( lter_kind=2 in &assim_tools_nml). Probability 0. 4 2 0 2 4. Mean of product is random sample of posterior. Product of random samples is random sample of product. Ensemble Filter Algorithms: Ensemble Kalman Filter (EnKF) ( lter_kind=2 in &assim_tools_nml). Probability 0. 4 2 0 2 4. Repeat this opera'on for each joint prior pair. Ensemble Filter Algorithms: Ensemble Kalman Filter (EnKF) ( lter_kind=2 in &assim_tools_nml). Probability 0. 4 2 0 2 4. Posterior sample maintains much of prior sample structure. (This is more apparent for larger ensemble sizes.)