Example: marketing

Factor Analysis - University of Minnesota

Factor AnalysisNathaniel E. HelwigAssistant Professor of Psychology and StatisticsUniversity of Minnesota (Twin Cities)Updated 16-Mar-2017 Nathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 1 CopyrightCopyrightc 2017 by Nathaniel E. HelwigNathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 2 Outline of Notes1) BackgroundOverviewFA vs PCA2) Factor Analysis ModelModel FormParameter EstimationFactor RotationFactor Scores3) Some ExtensionsOblique Factor ModelConfirmatory Factor Analysis4) Decathlon ExampleData OverviewEstimate Factor LoadingsEstimate Factor ScoresOblique RotationNathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 3 BackgroundBackgroundNathaniel E.

Factor Analysis Model Parameter Estimation Maximum Likelihood Estimation for Factor Analysis Suppose xi iid˘ N( ;LL0+ ) is a multivariate normal vector. The log-likelihood function for a sample of n observations has the form LL( ;L; ) = nplog(2ˇ) 2 + nlog(j n1j) 2 P i=1 (xi ) 0 1(x i ) 2 where = LL0+ . Use an iterative algorithm to maximize LL.

Tags:

  Analysis, Factors, Factor analysis, Estimation, Likelihood, Likelihood estimation

Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Other abuse

Transcription of Factor Analysis - University of Minnesota

1 Factor AnalysisNathaniel E. HelwigAssistant Professor of Psychology and StatisticsUniversity of Minnesota (Twin Cities)Updated 16-Mar-2017 Nathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 1 CopyrightCopyrightc 2017 by Nathaniel E. HelwigNathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 2 Outline of Notes1) BackgroundOverviewFA vs PCA2) Factor Analysis ModelModel FormParameter EstimationFactor RotationFactor Scores3) Some ExtensionsOblique Factor ModelConfirmatory Factor Analysis4) Decathlon ExampleData OverviewEstimate Factor LoadingsEstimate Factor ScoresOblique RotationNathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 3 BackgroundBackgroundNathaniel E.

2 Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 4 BackgroundOverviewDefinition and Purposes of FAFactor Analysis (FA) assumes the covariation structure among a set ofvariables can be described via a linear combination of unobservable(latent) variables called are three typical purposes of FA:1 Data reduction: explain covariation betweenpvariables usingr<platent factors2 Data interpretation: find features ( , factors ) that are importantfor explaining covariation (exploratory FA)3 Theory testing: determine if hypothesized Factor structure fitsobserved data (confirmatory FA)Nathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 5 BackgroundFactor Analysis versus Principal Components AnalysisDifference between FA and PCAFA and PCA have similar themes, , to explain covariation betweenvariables via linear combinations of other , there are distinctions between the two approaches:FA assumes a statistical model that describes covariation inobserved variables via linear combinations of latent variablesPCA finds uncorrelated linear combinations of observed variablesthat explain maximal variance (no latent variables here)FA refers to a statistical model, whereas PCA refers to the eigenvaluedecomposition of a covariance (or correlation) E.

3 Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 6 Factor Analysis ModelFactor Analysis ModelNathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 7 Factor Analysis ModelModel FormFactor Model withmCommon FactorsX= (X1,..,Xp) is a random vector with mean vector andcovariance matrix .The Factor Analysis model assumes thatX= +LF+ whereL={`jk}p mdenotes the matrix of Factor loadings`jkis the loading of thej-th variable on thek-th common factorF= (F1,..,Fm) denotes the vector of latent Factor scoresFkis the score on thek-th common Factor = ( 1,.., p) denotes the vector of latent error terms jis thej-th specific factorNathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 8 Factor Analysis ModelModel FormOrthogonal Factor Model AssumptionsThe orthogonal FA model assumes the formX= +LF+ and adds the assumptions thatF (0,Im), , the latent factors have mean zero, unit variance,and are uncorrelated (0, )where =diag( 1.)

4 , p)with jdenoting thej-thspecific variance jandFkare independent of one another for all pairsj,kNathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 9 Factor Analysis ModelModel FormOrthogonal Factor Model Implied Covariance StructureThe implied covariance structure forXisVar(X) =E[(X )(X ) ]=E[(LF+ )(LF+ ) ]=E[LF F L ] +E[LF ] +E[ F L ] +E[ ]=LE[F F ]L +LE[F ] +E[ F ]L +E[ ]=LL + whereE[F F ] =Im,E[F ] =0m p,E[ F ] =0p m, andE[ ] = .This implies that the covariance betweenXandFhas the formCov(X,F) =E[(X )F ]=E[(LF+ )F ] =LNathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 10 Factor Analysis ModelModel FormVariance Explained by Common FactorsThe portion of variance of thej-th variable that is explained by themcommon factors is called the communality of thej-th variable: jj Var(Xj)=h2j communality+ j uniquenesswhere jjis the variance ofXj( , thej-th diagonal of )h2j= (LL )jj=`2j1+`2j2+ +`2jmis the communality ofXj jis the specific variance (or uniqueness) ofXjNote that the communalityh2jis the sum of squared loadings E.

5 Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 11 Factor Analysis ModelParameter EstimationPrincipal Components Solution for Factor AnalysisNote that the parameters of interest are the Factor loadingsLandspecific variances on the diagonal of .Form<pcommon factors , the PCA solution estimatesLand as L=[ 1/21v1, 1/22v2,.., 1/2mvm] j= jj h2jwhere =V V is the eigenvalue decomposition of , and h2j= mk=1 `2jkis the estimated communality of thej-th of total sample variance explained by thek-th Factor isR2k= pj=1 `2jk pj=1 jj=( 1/2kvk) ( 1/2kvk) pj=1 jj= k pj=1 jjNathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 12 Factor Analysis ModelParameter EstimationIterated Principal Axis Factoring MethodAssume we are applying FA to a sample correlation matrixRR =LL and we have some initial estimate of the specific variance use j=1/rjjwhererjjis thej-th diagonal ofR 1 The iterated principal axis factoring algorithm:1 Form R=R given current jestimates2 Update L=[ 1/21 v1, 1/22 v2.]

6 , 1/2m vm]where R= V V is theeigenvalue decomposition of R3 Update j=1 mk=1 `2jkNathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 13 Factor Analysis ModelParameter EstimationMaximum likelihood estimation for Factor AnalysisSupposexiiid N( ,LL + )is a multivariate normal log- likelihood function for a sample ofnobservations has the formLL( ,L, ) = nplog(2 )2+nlog(| 1|)2 ni=1(xi ) 1(xi )2where =LL + . Use an iterative algorithm to of ML solution: there is a simple relationship between FAsolution forS(covariance matrix) andR(correlation matrix).If is the MLE of , theng( )is the MLE ofg( )Nathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 14 Factor Analysis ModelFactor RotationRotating Points in Two DimensionsSuppose we havez= (x,y) R2, , points in 2D Euclidean 2 2 orthogonal rotation of(x,y)of the form(x y )=(cos( ) sin( )sin( )cos( ))(xy)rotates(x,y)counter-clockwise around the origin by an angle of and(x y )=(cos( )sin( ) sin( )cos( ))(xy)rotates(x,y)clockwise around the origin by an angle of.

7 Nathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 15 Factor Analysis ModelFactor RotationVisualization of 2D Clockwise Rotationlllllllllll 3 2 10123 3 2 10123No Rotationxyabcdefghijklllllllllll 3 2 10123 3 2 10123xyrot[,1]xyrot[,2]abcdefghijk30 degreeslllllllllll 3 2 10123 3 2 10123xyrot[,1]xyrot[,2]abcdefghijk45 degreeslllllllllll 3 2 10123 3 2 10123xyrot[,1]xyrot[,2]abcdefghijk60 degreeslllllllllll 3 2 10123 3 2 10123xyrot[,1]xyrot[,2]abcdefghijk90 degreeslllllllllll 3 2 10123 3 2 10123xyrot[,1]xyrot[,2]abcdefghijk180 degreesNathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 16 Factor Analysis ModelFactor RotationVisualization of 2D Clockwise Rotation (R Code)rotmat2d <- function(theta){matrix(c(cos(theta),sin( theta),-sin(theta),cos(theta)),2,2)}x <- seq(-2,2,length=11)y <- 4*exp(-x^2) - 2xy <- cbind(x,y)rang <- c(30,45,60,90,180) (width=12,height=8,noRStudioGD=TRUE)par( mfrow=c(2,3))plot(x,y,xlim=c(-3,3),ylim= c(-3,3),main="No Rotation")text(x,y,labels=letters[1:11], cex= )for(j in 1:5){rmat <- rotmat2d(rang[j]*2*pi/360)xyrot <- xy%*%rmatplot(xyrot,xlim=c(-3,3),ylim=c( -3,3))text(xyrot,labels=letters[1:11],ce x= )title(paste(rang[j]," degrees"))}Nathaniel E.

8 Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 17 Factor Analysis ModelFactor RotationOrthogonal Rotation in Two DimensionsNote that the 2 2 rotation matrixR=(cos( ) sin( )sin( )cos( ))is an orthogonal matrix for all :R R=(cos( )sin( ) sin( )cos( ))(cos( ) sin( )sin( )cos( ))=(cos2( ) +sin2( )cos( )sin( ) cos( )sin( )cos( )sin( ) cos( )sin( )cos2( ) +sin2( ))=(1 00 1)Nathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 18 Factor Analysis ModelFactor RotationOrthogonal Rotation in Higher DimensionsSuppose we have a data ofXare coordinates of points inp-dimensional spaceNote: whenp=2 we have situation on previous slidesAp porthogonal rotation is an orthogonal linear R=RR =IpwhereIpisp pidentity matrixIf X=XRis rotated data matrix, then X X =XX Orthogonal rotation preserves relationships between pointsNathaniel E.

9 Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 19 Factor Analysis ModelFactor RotationRotational Indeterminacy of Factor Analysis ModelSupposeRis an orthogonal rotation matrix, and note thatX= +LF+ = + L F+ where L=LRare the rotated Factor loadings F=R Fare the rotated Factor scoresNote that L L =LL , so we can orthogonally rotate the FA solutionwithout changing the implied covariance E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 20 Factor Analysis ModelFactor RotationFactor Rotation and Thurstone s Simple StructureFactor rotation methods attempt to find some rotation of a FA solutionthat provides a more parsimonious s (1947) simple structure describes an ideal Factor solution1 Each row ofLcontains at least one zero2 Each column ofLcontains at least one zero3 For each pair of columns ofL, there should be several variableswith small loadings on only one of the two factors4 For each pair of columns ofL, there should be several variableswith small loadings on both factors ifm 45 For each pair of columns ofL, there should be only a few variableswith large loadings on both factorsNathaniel E.

10 Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 21 Factor Analysis ModelFactor RotationOrthogonal Factor Rotation MethodsMany popular orthogonal Factor rotation methods try to maximizeV(L,R| ) =1pm k=1 p j=1( `jk/ hj)4 p p j=1( `jk/ hj)2 2 where `jkis the rotated loading of thej-th variable on thek-th Factor hj= mk=1 `2jkis the square-root of the communality forXjChanging the parameter corresponds to different criertia =1 corresponds to varimax criterion =0 corresponds to quartimax criterion =m/2 corresponds to equamax criterion =p(m 1)/(p+m 2)corresponds to parsimax criterionNathaniel E. Helwig (U of Minnesota ) Factor AnalysisUpdated 16-Mar-2017 : Slide 22 Factor Analysis ModelFactor ScoresIssues Related to Factor ScoresIn FA, one may want to obtain estimates of the latent Factor ,Fis a random variable, so estimating realizations ofFisdifferent from estimating the parameters of the FA model (Land ).


Related search queries