Lecture 13 Principal Components Analysis and Factor ... - KIT

1 Lecture 13 Principal Components Analysis and FactorAnalysisProf. Dr. Svetlozar RachevInstitute for Statistics and Mathematical EconomicsUniversity of KarlsruheFinancial Econometrics, Summer Semester 2007 Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical Economics University of KarlsruheLecture 13 Principal Components Analysis and Factor AnalysisCopyrightThese Lecture -notes cannot be copied and/or distributedwithout material is based on the text-book:Financial Econometrics: From Basics to AdvancedModeling Techniques(Wiley-Finance, Frank J. Fabozzi Series)by Svetlozar T. Rachev, Stefan Mittnik, Frank Fabozzi, SergioM. Focardi,TeoJa s i` Dr. Svetlozar Rachev Institute for Statistics and Mathematical Economics University of KarlsruheLecture 13 Principal Components Analysis and Factor AnalysisOutlineIFactor Components and Factor Analysis Dr.

2 Svetlozar Rachev Institute for Statistics and Mathematical Economics University of KarlsruheLecture 13 Principal Components Analysis and Factor AnalysisFactor ModelsIFactor models are statistical models that try to explaincomplex phenomena through a small number of basic causesor models serve two main purposes:1. They reduce the dimensionality of models to makeestimation possible;2. They find the true causes that drive models were introduced by Charles Spearman in Dr. Svetlozar Rachev Institute for Statistics and Mathematical Economics University of KarlsruheLecture 13 Principal Components Analysis and Factor AnalysisFactor ModelsIThe Spearman model explains intellectual abilities through onecommon Factor , the famous general intelligence gfactor,plus another factorswhich is specific to each distinct Leon Thurstone developed the first true multifactormodel of intelligence, where were identified the followingseven primary mental abilities:Verbal ComprehensionWord FluencyNumber FacilitySpatial VisualizationAssociative MemoryPerceptual Dr.

3 Svetlozar Rachev Institute for Statistics and Mathematical Economics University of KarlsruheLecture 13 Principal Components Analysis and Factor AnalysisFactor ModelsIIn the early applications of Factor models to psychometrics,the statistical model was essentially a conditional multivariatedistribution. The objective was to explain psychometric testsas probability distributions conditional on the value of one ormore factors. In this way, one can make predictions of, forexample, the future success of young individuals in economics, Factor models are typically applied to timeseries. The objective is to explain the behavior of a largenumber of stochastic processes, typically price, returns, or rateprocesses, in terms of a small number of factors.

4 Thesefactors are themselves stochastic Dr. Svetlozar Rachev Institute for Statistics and Mathematical Economics University of KarlsruheLecture 13 Principal Components Analysis and Factor AnalysisFactor ModelsIIn order to simplify both modeling and estimation, most factormodels employed in financial econometrics are static means that time series are assumed to be sequences oftemporally independent and identically distributed (IID)random variables so that the series can be thought asindependent samples extracted from one common financial econometrics, Factor models are needed not onlyto explain data but to make estimation feasible. Factormodels able to explain all pairwise correlations in terms of amuch smaller number of correlations between Dr.

5 Svetlozar Rachev Institute for Statistics and Mathematical Economics University of KarlsruheLecture 13 Principal Components Analysis and Factor AnalysisLinear Factor Models EquationsLinear Factor models are regression models of the following type:Xi= i+K j=1 ijfj+ iwhereXi= a set ofNrandom variablesfj= a set ofKcommon factors i= the noise terms associated with each variableXi ij s are thefactor loadingsorfactor sensitivities, which expressthe influence of thej-th Factor on thei-th : In this formulation, Factor models are essentially staticmodels, but it is possible to add a dynamics to both the variablesand the Dr. Svetlozar Rachev Institute for Statistics and Mathematical Economics University of KarlsruheLecture 13 Principal Components Analysis and Factor AnalysisFactor ModelsIOne of the key objectives of Factor models is that thecovariances between the variablesXiis determined only by thecovariances between that the noise terms are mutually uncorrelated, sothatE( i j) ={0i6=j 2ii=jand that the noise terms are uncorrelated with the factors,that is,E( ifj) = 0, i, also that both factors and noise terms have a zeromean, so thatE(Xi) = models that respect the above constraints are calledstrictfactor Dr.}

6 Svetlozar Rachev Institute for Statistics and Mathematical Economics University of KarlsruheLecture 13 Principal Components Analysis and Factor AnalysisFactor ModelsLets compute the covariances of a strict Factor model:E((Xi i)(Xj j)) =E((K s=1 isfs+ i)(K t=1 jtft+ j))=E((K s=1 isfs)(K t=1 jtft))+E((K s=1 isfs)( j))+E(( i)K t=1 jtft)+E( i j)= s,t isE(fsft) jt+E( i j)Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical Economics University of KarlsruheLecture 13 Principal Components Analysis and Factor AnalysisFactor ModelsWe can express the above compactly in matrix form. Lets write afactor model in matrix form as follows:X= + f+ whereX= (X1,..,XN) = theN-vector of variables = ( 1.)

7 , N) = theN-vector of means = ( 1,.., N) = theN-vector of idiosyncratic noise termsf= (f1,..,fK) = theK-vector of factors = 11 N1 NK =theN Kmatrix of Factor Dr. Svetlozar Rachev Institute for Statistics and Mathematical Economics University of KarlsruheLecture 13 Principal Components Analysis and Factor AnalysisFactor ModelsILets define the following: = theN Nvariance-covariance matrix of the variablesX = theK Kvariance-covariance matrix of the factors =N Nvariance-covariance matrix of the error terms .IIf we assume that our model is a strict Factor model, thematrix will be a diagonal matrix with the noise variances onthe diagonal, that is, = 21 2N IWe can express the variance-covariance matrix of the variablesin the following way: = + Prof.

8 Dr. Svetlozar Rachev Institute for Statistics and Mathematical Economics University of KarlsruheLecture 13 Principal Components Analysis and Factor AnalysisFactor ModelsIIn applied work, Factor models will often be approximatefactor models. They allow idiosyncratic terms to be weaklycorrelated among themselves and with the many different Factor models have been proposed forexplaining stock returns, an important question is whether afactor model is fully determined by the observed time estimation procedure cannot univocally determine thehidden factors and the Factor loadings from the Dr. Svetlozar Rachev Institute for Statistics and Mathematical Economics University of KarlsruheLecture 13 Principal Components Analysis and Factor AnalysisFactor ModelsIn fact, suppose that we multiply the factors by any nonsingularmatrixR.

9 We obtain other factorsg=Rfwith a covariance matrix g=R R 1and we can write a new Factor model:X= + f+ = + R 1Rf+ = + gg+ Prof. Dr. Svetlozar Rachev Institute for Statistics and Mathematical Economics University of KarlsruheLecture 13 Principal Components Analysis and Factor AnalysisFactor ModelsIIn order to solve this indeterminacy, we can always choose thematrixRso that the factorsgare a set of orthonormalvariables, that is, uncorrelated variables (the orthogonalitycondition) with unit variance (the normality condition).IIn order to make the model uniquely identifiable, we canstipulate that factors must be a set of orthonormal variablesand that, in addition, the matrix of Factor loadings is this additional assumption, a strict Factor model iscalled anormal Factor model.

10 The model is still undeterminedunder rotation, that is multiplication by any nonsingularmatrix such thatRR = Dr. Svetlozar Rachev Institute for Statistics and Mathematical Economics University of KarlsruheLecture 13 Principal Components Analysis and Factor AnalysisFactor ModelsSummary:IA set of variables has a normal Factor representation if it isrepresented by the following Factor model:X= + f+ where factors are orthonormal variables and noise terms aresuch that the covariance matrix can be represented as follows: = + where is the diagonal matrix of Factor loadings and is adiagonal Factor models are uniquely identifiable only inthe limit of an infinite number of Dr. Svetlozar Rachev Institute for Statistics and Mathematical Economics University of KarlsruheLecture 13 Principal Components Analysis and Factor AnalysisFactor Models: Types of Factors and Their EstimationIn financial econometrics, the factors used in Factor models canbelong to three different categories:IMacroeconomic factorsIFundamental factorsIStatistical factorsMacroeconomic factorsare macroeconomic variables that arebelieved to determine asset returns (Example: GNP, the inflationrate, the unemployment rate, or the steepness of the yield curve).