潜在トピックモデルに ... - latent-dynamics.net

1 1 2 3 D. Blei, A. Ng, M. Jordan, Latent Dirichlet Allocation, JMLR2003 4 D. Blei, J. Lafferty, Dynamic Topic Models, ICML20065 [L. Cao and L. Fei-Fei. Spatially coherent latent topic model for concurrent object segmentation and classification . ICCV2007]6 7bag-of-word W W 1w2w3w4w5w6w7w8w w8 1()( )NnnPPw== w w ()Pw9 1()() (|)NnznPPzPw z== w (|)Pw z ()Pzzw 10 1()(|)( |)NnznPPz Pw z == w (|)Pw z(|)Pz zw 11 PLSA LDA PLSA =PLSI : Probabilistic Latent Semantic analysis LDA: Latent Dirichlet Allocation PLSA+ 1() ()(|)( |)NnznPPPzPwzd == w1()(|)( |)NnznPPz Pw z == w 12 D.

2 Blei, A. Ng, M. Jordan, Latent Dirichlet Allocation, JMLR2003 LDAPLSA13 w1D wND zwND zK K wND wN= 14 Collapsed ()\\\\(|,)nkw nnndknknNPz kNNV += + +WZ \,nwz d k k wn n wNDzK 15 = =visual word = = ..SIFTk-means2225321016 [Wang and McCallum, 06] [Li and McCallum, 06] [Rosen-Zvi, et al. 04] [Blei and Lafferty, 07] 17 T. Iwata, T. Yamada, N. Ueda, Probabilistic Latent Semantic Visualization: Topic Model for Visualizing Documents, KDD2008 T. Iwata, S. Watanabe, T. Yamada, N. Ueda, "Topic Tracking Model for Analyzing Consumer Purchase Behavior, IJCAI2009 T. Iwata, T. Yamada, N. Ueda, "Modeling Social Annotation Data with Content Relevance using a Topic Model," NIPS2009 T.

3 Iwata, T. Yamada, Y. Sakurai, N. Ueda, "Online Multiscale Dynamic Topic Models, KDD201018 T. Iwata, S. Watanabe, T. Yamada, N. Ueda, "Topic Tracking Model for Analyzing Consumer Purchase Behavior, IJCAI200919 Introduction 20 Purchase behavior model(| ,)Pi utitemtimeuser t u i ABC Michael JacksonBeatlesDeep purpleMadonna1982itemsusershighlow 21 Purchase behavior model(| ,)Pi utitemtimeuserABC Michael JacksonBeatlesDeep purpleMadonna1993 t u i 22 Purchase behavior model(| ,)Pi utitemtimeuserABC Michael JacksonBeatlesDeep purpleMadonna2009 t u i 23 Purchase behavior model(| ,)Pi utitemtimeuserABC Michael JacksonBeatlesDeep purpleMadonna2009 t u i 24 Topic model Probabilistic latent semantic analysis (| ,)( | ,) (| ,)tzu tzizzPi utPz utPi zt == M.

4 JacksonBeatlesDeep PurpleMadonnaPopR&BHeavy Metal usertopic itemtopicM. JacksonMadonnaColdplayDeep PurpleHelloweenRoyal Hunt= useriteminterests PopMetaltrend 25 Latent Dirichlet Allocation (LDA) (| ,)tzu tzizPi ut = 1()tutuzzP 1()tztziiP Interest priortrend priorPopR&BMetalPopR&BMetalPopR&BMetalA, tA,t+1 PopR&BMetalB,tBayesian extension26 Latent Dirichlet Allocation (LDA) (| ,)tzu tzizPi ut = 1()tutuzzP 1()tztziiP Interest priortrend priorPopR&BMetalPopR&BMetalPopR&BMetalA, tA,t+1 PopR&BMetalB,t Bayesian extension TTM Topic Tracking Model (TTM)28 Incorporate the dynamics of interests (Dirichlet) ,1,,,, (| ,)tut ututu zzP mean: previous interests t-1,uprecision: t,u PopR&BMetaltt+1t+2 User.

5 UInterest prior,1,, 1tu t uz t,u t+1,u t+2,u t+3,u29 Incorporate the dynamics of trends +1t+2 Popmean: previous trends t-1,zprecision: t,z ,1,,,, (| ,)tzt ztztziiP Trend prior,1,, 1tz t zi t,z t+1,z t+2,z t+3,z30 Online inference EM Interests Purchase logs XTrends Persistency , , X , Xtt+1t-1a sample (user, item, time)31 Stochastic EME-step: collapsed Gibbs sampling of topic zM-step: maximum likelihood of , Estimate and after the iteration of EM steps32 Capturing long term dependences L PopR&BMetalt-2t-1tUser: u t,A,1 t,u,2 t,u,3 33 Graphical models of TTM and related modelsxUM z ZxM z ZDynamicMixtureModelDynamicTopic ModelGaussianGaussianlatent topicpurchased itemItem trends# purchased items# topicsxUM z ZxUM z Z# usersTrend persistencyUser interestsInterest persistencyt-1t[Blei et al.]

6 , ICML06][Wei et al., IJCAI07]34 Experiments Movie 110 days, 70,122 users, 7,469 items 11,243,935 transactions Cartoon 151 days, 143,212 users, 206 items 12,642,505 transactions LDAall LDAonline LDA LDAone 35 Average N-best accuracies (%)36 Average computational time (sec) LDAall TTM10 1 100,000 12 TTM 37 Examples of trend persistency for each day time 38 Trend analysis in Delicious tags Delicious 2005/11 2008/3) (user, tag, time) 37 , 15,466 , 14,586 2008/11 politics,obama,election,election2008,mcc ainmobile,iphone,google,microsoft, ,twitter,socialmedia,tools,socialnetwork ingmac,iphone,osx,software,apple39 Topic politics 2007/8 politics,economics,history,news,culture2 007/9 politics,economics,history,news,war2007/ 10 politics,economics,history,news,law2007/ 11 politics,economics,history,media,news200 7/12 politics,economics,history,religion,gove rnment2008/1 politics,economics,history,government,me dia2008/2 politics,economics,history,obama,governm ent2008/3 politics,economics,history,obama,war2008 /4 politics,economics,history,media,governm ent2008/5 politics,economics,history,culture,gover nment2008/6 politics,obama,economics,history,law2008 /7 politics,economics,obama,history,law2008 /8 politics,obama,economics,history,war2008 /9 politics,mccain,palin,election,obama2008 /10 politics,obama,election,mccain,economics 2008/11 politics,obama,election.

7 Election2008,mccainJune/3/2008 Obama was named the presumptive McCain announced that he had chosen Palin as his running the Presidential Election40 Topic 2007/1 ,social,community,search,blogging,google ,video,internet,secondlife,rss2007/2 ,community,social,video,rss,blogging,goo gle,search,tools,yahoo2007/3 ,community,social,blogging,twitter,colla boration,google,search,rss,video2007/4 ,community,social,blogging,video,google, twitter,collaboration,rss,search2007/5 ,community,social,search,video,google,bl ogging,twitter,tools,socialnetwork2007/6 ,community,social,video,facebook,bloggin g,tools,google,socialnetworking,s2007/7 ,facebook,community,socialnetworking,soc ial,blogging,tools,video,google, 2007/8 ,social,socialnetworking,facebook,commun ity,video,blogging,google,tools,s2007/9 ,facebook,socialnetworking,social,google ,community,tools,blogging,video,c2007/10 ,facebook,socialnetworking,social,google ,community,blogging,video,tools2007/11 ,facebook,socialnetworking,social,google ,community,advertising,opensoc2007/12 ,facebook,socialnetworking,social,google ,blogging,twitter,community,vide2008/1 ,socialnetworking,facebook,twitter,socia l,socialmedia,blogging,tools,comm2008/2 ,socialnetworking,twitter,social,communi ty,google,socialmedia,facebook,to2008/3 ,twitter,socialnetworking,social,socialm edia,community,blogging,facebook,2008/4 ,twitter,socialnetworking,social,socialm edia,blogging,tools,community,face2008/5 ,twitter,socialnetworking,social,socialm edia,community,tools,blogging,face2008/6 ,twitter,socialnetworking,socialmedia,so cial,tools,blogging,community,face2008/7 ,twitter,socialmedia,socialnetworking,so cial,tools,blogging,community,goog2008/8 ,twitter,tools,socialmedia,socialnetwork ing,social,blogging,search,facebook2008/ 9 ,twitter,socialmedia,socialnetworking,to ols,social,google,blogging,search,tr2008 /10 ,twitter.

8 Socialmedia,tools,socialnetworking,socia l,blogging,marketing,goog2008/11 ,twitter,socialmedia,tools,socialnetwork ing,social,facebook,microblogging41 Teh et al., Hierarchical Dirichlet processes, Journal of the American Statistical Association, 2006 Morinaga and Yamanishi, Tracking Dynamics of Topic Trends using a Finite Mixture Model, KDD2004 Wang et al., Continuous Time Dynamic Topic Models, UAI2008 Iwata et al., Online Multiscale Dynamic Topic Models, KDD201042