Time-Inconsistency: Problems and Mathematical Theory

1 time - inconsistency : Problems and Mathematical TheoryJiongmin Yong(University of Central Florida)December 28, 2015 Outline1. Introduction: time -Consistency2. time -Inconsistent Problems3. Equilibrium Strategies4. Open Problems1. Introduction: time -ConsistencyContinuous Compound Interest Exponential (0) initial principalr annual interest rateP(t) =P(0)ert Amount at the end oft-th year(compounded continuously)For any given future timesT>t>0, fromP(T) =P(0)erT,P(t) =P(0)ert,one hasP(T) =P(t)er(T t),0<t<T,or, equivalently,P(t) =P(T)e r(T t),0<t< is the value (price) attof a payoffP(T) r(T t) exponential any 0<t1<t2<T, one haser(T t1)P(t1) =P(T) =er(T t2)P(t2).Therefore,possessP(t1) att1 possessP(t2) att2 This is called theTime-Consistencyofexponential discountingPreferred Choice:Assume that annual rate isr= 10%Option(A): Get $100 today (December 28, 2015).

2 Option(B): Get $105 (>100(1 +r12)) on January 28, (A ): Get $110 (= 100 ) on December 28, (B ): Get $ (>110(1 +r12)) on January 28, atime-consistentperson,(A) (A ),(B) (B ),(B) (A),(B ) (A ).(We will come back to this example later)Semigroups:Consider X(s) =b(s,X(s)),s [t,T],X(t) = for any(t,x) [0,T) Rn, the above admits a uniquesolutionX( ;t,x). Then for any (t,T),X(s; ,X( ;t,x)) =X(s;t,x),s [ ,T].The restrictionX( ;t,x) [ ,T]is the solution of the equationstarting from( ,X( ;t,x)).A (nonlinear) semigroup Programming/Feymann-Kac formula:Consider X(s) =b(s,X(s)),s [0,T],J(t,X(t)) =h(X(T)) + Ttg(s,X(s)) (t,T),J(t,X(t)) =h(X(T;t,X(t))) + Ttg(s,X(s;t,X(t)))ds=h(X(T; ,X( ;t,X(t)))) + T g(s,X(s; ,X( ;t,X(t))))ds+ tg(s,X(s;t,X(t)))ds=J( ,X( )) + tg(s,X(s)) semigroup property.]

3 (Special case:h(x) =x,g= 0)This leads to Jt(t,x) +Jx(t,x)b(t,x) +g(t,x) = 0,J(T,x) =h(x).(1)LinearHamilton-Jacobitype viewpoint:The solutionJ(t,x)of PDE (1) admitsrepresentation:J(t,x) =h(X(T;t,x)) + Ttg(s,X(s;t,x)) is a deterministicFeynman-Kac Control problem :Consider X(s) =b(s,X(s),u(s)),s [t,T],X(t) =x,with (scalar) cost functionalJ(t,x;u( )) =h(X(T)) + Ttg(s,X(s),u(s))ds,whereU[t,T] ={u: [t,T] U u( )is measurable}. problem (C).For given(t,x) [0,T) Rn, find u( ) U[t,T]such thatJ(t,x; u( )) =infu( ) U[t,T]J(t,x;u( )) V(t,x).Bellman Optimality Principle:For any [t,T],V(t,x) =infu( ) U[t, ][ tg(s,X(s),u(s))ds+V( ,X( ;t,x,u( )))].Let( X( ), u( ))be optimal for(t,x) [0,T) (t,x) =J(t,x; u( )) = tg(s, X(s), u(s))ds+J( , X( ;t,x, u( )); u( ) [ ,T]) tg(s, X(s), u(s))ds+V( , X( ;t,x, u( ))) infu( ) U[t, ] tg(s,X(s),u(s))ds+V( ,X( ;t,x,u( )))=V(t,x).]]

4 Thus, all the equalities ,J( , X( ); u( ) [ ,T])=V( , X( ))=infu( ) U[ ,T]J( , X( );u( )), , u( ) [ ,T] U[ ,T] isoptimalfor( , X( ;t,x, u( ))).This is called thetime-consistencyof problem (C). problem involving a decision-making is said to betime-consistentifanoptimaldecision made at a given timetwill remainoptimalat any times> the above is not the case, the problem is said to betime-inconsistent.**If the problem under consideration is time -consistent, then oncean optimal decision is made, we will not regret afterwards!If the whole world istime-consistent,then the things are tooideal, the life will be mucheasier!But, it might also be a little or tooboring(exciting to have some challenges)!Fortunately (unfortunately?)

5 , the life is not that ideal!(Challenges are around!) time -inconsistent Problems exist almost everywhere!2. time -Inconsistent ProblemsIn reality, Problems arehardly time -consistent:An optimal decision/policy made at timet, more than often,will not stay optimal, reason:When building the model, describing theutility/cost, etc., the following are used:subjective time -Preferencesandsubjective Risk-Preferences. time -Preferences:Most people do not discount exponentially! Instead, they overdiscount on the utility of immediate future outcomes.* Overreaction without thinking the consequences(bad temper and impatience lead to unnecessary fighting,..)* Break promise, delay planned projects (fail to meet deadlines,such as refereeing papers, quit smoking.)

6 * Shopping using credit cards (meeting immediate satisfaction,big discount, buy one get one free,..)* Unintentionally pollute the environment due to over-development* Corruption, without thinking things not because you need to dobut because you like to Doing things not because you do not need to dobut because you do not like to do.* D. Hume (1739), A Treatise of Human Nature Reason is, and ought only to be the slave of the passions. More than often, people doing things is due to theirpassions.* A. Smith (1759), The Theory of Moral Sentiments Utility is not intertemporally sparable but rather thatpast and future experiences, jointly with current ones,provide current , in Mathematical terms, one should haveU(t,X(t)) =f(U(t r,X(t r)),U(t+ ,X(t+ )),whereU(t,X) is the utility at (t,X).)

7 Exponential discounting: e(t) =e rt,r>0 discount rateHyperbolic discounting: h(t) =11+kt a hyperbolaIf letk=er 1, ,e r= e(1) = h(1) =11+k, then e(t) =e rt=1(1 +k)t, h(t) =11 + 0,t7 11+ktdecreases faster thant7 1(1+k)t: h(0) = k< ln(1 +k) = e(0),Hyperbolic discountingactually appears inpeople s back to a previous example:Annual rate is 10%Option(A): Get $100 today (December 28, 2015).Option(B): Get $105 (>100(1 +r12)) on January 28, (A ): Get $110 (= 100 ) on December 28, (B ): Get $ (>110(1 +r12)) on January 28, atime-consistentperson,(A) (A ),(B) (B ),(B) (A),(B ) (A ).However, for anuncertainty-averseperson,(A) (B),(B ) (A ).Magnifying the example:Option(A): Get $1M today (December 28, 2015).Option(B): Get $ (>1M(1 +r12)) on January 28, (A ): Get $ (= 1M ) on December 28, (B ): Get $ (> (1+r12)) on January 28, anuncertainty-averseperson,(A) (B),(B ) (A ).

8 The feeling is stronger?* Palacious Huerta (2003), survey on history* Strotz (1956), Pollak (1968), Laibson (1997), ..* Finn E. Kydland and Edward C. Prescott, (1977)(2004 Nobel Prize winners)(classical optimal control Theory not working)* Ekeland Lazrak (2008), Yong (2011, 2012) Risk-Preferences:Consider two investments whose returns are:R1andR2withP(R1= 100) =12,P(R1= 50) =12,P(R2= 150) =13,P(R2= 60) = one you prefer?ER1=12100 +12( 50) = 25,ER2=13150 +23( 60) = to be better.* St. Petersburg Paradox: (posed by Nicolas Bernoulli in 1713)P(X= 2n) =12n,n 1,E[X] = n=12nP(X= 2n) = n=12n12n= .Question:How much are you willing to pay to play the game?How about $10,000? Or $1,000? Or ???In 1738, Daniel Bernoulli (a cousin of Nicolas) introducedexpected utility:E[u(X)].

9 Withu(x) = x, one hasE X= n=1(1 2)n= 1 + 2.* 1944, von Neumann Morgenstern: Introduced rationality axioms: Completeness, Transitivity, Independence, stochastic optimal control Theory is based on theexpected utility Theory . Decision-making based on expected utility Theory istime-consistent. In classical expected utility Theory , the probability isobjective. It is controversial whether a probability should beobjective. Early relevant works: Ramsey (1926), de Finetti (1937)Allais Paradox (1953). ={1,2, ,100},P( ) =1100, .X1( ) = 100 {1 100},X2( ) = 200 {1 70},X3( ) = 100 {1 15},X4( ) = 200 {1 10}.Most people have the following preferences:X2 X1,X3 there exists a utility functionu:R R+such thatX Y E[u(X)]<E[u(Y)],thenX2 X1 E[u(X2)] = (200)<u(100) =E[u(X1)],X3 X4 E[u(X3)] = (100)< (200) =E[u(X4)],Thus, (100)< (200)<u(100), a s Paradox (1961).

10 In an urn, there are 90 balls,3060 RedBlackWhiteXR$10000XB0$1000XR W$1000$100XB W0$100$100 Most people have the following preferences: (ambiguity-averse)XB XR,XR W XB (R) =13,P(B) [0,23],P(B W) =23,P(R W) [13,1].P(B W) =P(B) +P(W),P(R W) =P(R) +P(W).XB XR,XR W XB (R) =13,P(B) [0,23],P(B W) =23,P(R W) [13,1].P(B W) =P(B) +P(W),P(R W) =P(R) +P(W).If there exists a utility functionu:R R+such thatX Y E[u(X)]<E[u(Y)],thenXR W XB W u(100)P(R W)<u(100)P(B W) u(100)P(R) =u(100)[P(R W) P(W)]<u(100)[P(B W) P(W)] =u(100)P(B) XR XB,a Literature:* Subjective expected utility Theory (Savage 1954)* Mean-variance preference (Markowitz 1952)leading to nonlinear appearance of conditional expectation* Choquet integral (1953)leading to Choquet expected utility Theory * Prospect Theory (Kahneman Tversky 1979)(Kahneman won 2002 Nobel Prize)* Distorted probability (Wang Young Panjer 1997)widely used in insurance/actuarial science* BSDEs, g-expectation (Peng 1997)leading to time -consistent nonlinear expectation* BSVIEs (Yong 2006,2008)leading to time -inconsistent dynamic risk measureRecent Relevant Literatures:* Bj ork Murgoci (2008), Bj ork Murgoci Zhou (2013)* Hu Jin Zhou (2012, 2015)* Yong (2012, 2013, 2014, 2015) A Summary: time -Preferences:(Exponential/Ge neral).