Understanding Semantic Change of Words Over …

1 Understanding Semantic Change of Words Over CenturiesDerry Tanti WijayaLanguage Technologies InstituteCarnegie Mellon University5000 Forbes , PA YeniterziLanguage Technologies InstituteCarnegie Mellon University5000 Forbes , PA this paper, we propose to model and analyze changesthat occur to an entity in terms of changes in the wordsthat co-occur with the entity over time. We propose to doan in-depth analysis of how this co-occurrence changes overtime, how the Change influences the state ( Semantic , role)of the entity, and how the Change may correspond to eventsoccurring in the same period of time. We propose to iden-tify clusters of topics surrounding the entity over time us-ing Topics-Over-Time (TOT) and k-means clustering.

2 Weconduct this analysis on Google Books Ngram dataset. Weshow how clustering Words that co-occur with an entity ofinterest in 5-grams can shed some lights to the nature ofchange that occurs to the entity and identify the period forwhich the Change occurs. We find that the period identifiedby our model precisely coincides with events in the sameperiod that correspond to the Change that and Subject [Information Systems]: Database Management Database Applications, Data Mining; [Information Sys-tems]: Information Storage and Retrieval; [NaturalLanguage Processing]: Text analysis topics over time,k-means clusteringGeneral TermsAlgorithms, clustering, topic transition over time, Semantic Change ,event ; Words or real world entities Change over Change semantically and this Change is reflected in theway the Words are being used.

3 As people use Words in newPermission to make digital or hard copies of all or part of this work forpersonal or classroom use is granted without fee provided that copies arenot made or distributed for profit or commercial advantage and that copiesbear this notice and the full citation on the first page. To copy otherwise, torepublish, to post on servers or to redistribute to lists, requires prior specificpermission and/or a 11,October 24, 2011, Glasgow, Scotland, 2011 ACM 978-1-4503-0962-2/11/10 ..$ , the meanings of the Words Change gradually, oftento the point that the new meaning is radically different fromthe original usage.

4 For example, awful1originally meant awe-inspiring, filling someone with deep awe , as inthe awfulmajesty of the Creator. At some point it becomes somethingextremely bad, as in an awfully bad performance, but nowthe intensity of the expression has lessened and the word isnow used informally to just mean very bad , as inan awfulmess. Some Words also Change semantically, not in theiroriginal meanings but Change in a way that they acquireadditional meanings or are used to refer to other namedentities over time. For example, mouse is used originally torefer to small long-tailed animal but it is now also used torefer to a device used to control cursor identifying changes to an entity over time isbeneficial to many natural language applications.

5 For exam-ple, for a macro-reader2that gathers background/common-sense facts about entities from a large collection of inputtext, it is important for the reader to automatically identifytemporal changes that occur to the entities since it will mo-tivate a time-aware and hence a more precise possible application of Change identification isevent extraction. This is a difficult problem in informa-tion extraction because unlike other named-entities, at thewords/sentences (surface-) level, it is difficult to assign aprecise label to an event. Furthermore, an event usuallyinvolves multiple entities and multiple relations and thereare multiple ways to express the same event.

6 By modelingan event at a meta-level as a sequence of topics Change overtime, we can extract more similar events from a collection oftexts when a similar sequence of topics Change is Change over time in the frequency of entity men-tions in a collection of texts can indicate that there is achange happening to the entity when its frequency , a Change to an entity may not always be accom-panied by a Change in its frequency. Furthermore, knowingthat there is a frequency Change does not give further insightto the nature of the Change or what causes this paper, we propose to conduct a deeper analysisthan frequency Change , by learning about the nature of thechange itself from the Change in the Words that co-occur withthe entity over time.

7 In the next section we describe relatedworks in this area and describe our proposed approach inSection 3. We describe our experimental setting in Section4 and give detailed analysis of our experiment results inSection 5. We conclude with future works in Section kemmer/Words04/meaning2 NELL: WORKSA quantitative study on cultural trends: culturomics that focuses on linguistic and cultural phenomena was donewith the computational analysis of Google Books Ngramdataset [3]. By studying usage frequency over time of the n-grams that represent entities of interest, they study linguis-tic changes, lexicon and grammar changes; and culturalphenomena, how people and events are terms of grammatical trends, they study the frequencychange in English irregular verbs.

8 They find that high-frequency irregulars such as found are less likely to be re-placed by their regular forms than lower frequency irregularssuch as dwelt . In terms of cultural phenomena, they trackfame by measuring the frequency of a famous person s find that famous people in different periods of time fol-low the same kind of trajectories: pre-celebrity period, rapidrise to prominence, a peak and a slow decline in fame. Byfollowing such trajectories, one might be able to identifystatus of famous people in different periods of time. Theyconclude by highlighting that culturomics challenge liesinthe interpretation of evidence (in this case, frequency) pro-vided by the large Google dataset.

9 Our paper intends tocomplement their interpretation further by using not onlyfrequency but also actual Words and word co-occurrences inthe n-grams to study linguistic and cultural Change . For ex-ample, the authors hypothesize that the Change in the word speed from its irregular form: sped to its regular form: speeded might have been caused by the shift in meaningfrom to move rapidly towards to exceed the legal limit , to speed up [3]. The purpose of our paper is preciselyto enable us to confirm or refute such another work, the authors study temporal changes inpublic opinion in tweets [1]. They identify a Change (a break-point) in public opinion when there are both emotion patternand word pattern Change (measured with cosine and Jaccardsimilarity changes) in tweets from one point in time to an-other.

10 If a pattern in a time period is less similar to a patternin the preceding period but more similar to a pattern in thefollowing period, a breakpoint is reported and events thatcause this Change are described by choosing keywords fromall tweets in that period of Change . Unlike this paper, ourpaper uses a Change in topic surrounding an entity as an in-dication of Change to the entity thus directly finds keywordsthat describe the Change from the topics Pre-processing StepFor a given wordwthat represents an entity of interest( awful , mouse ), we retrieve all 5-grams that containw(case insensitive) as its third word. We are interested inthe two Words beforewand the two Words afterwin the5-grams.