Towardalinguisticallyrealisticassessmentoflanguage ...

1 Vol. 11 (2017), pp. 103 113 Version Received: 16 Jan 2017 Toward a linguistically realistic assessment of languagevitality:The case of JejueoChangyong YangJeju National UniversityWilliam O GradyUniversity of Hawai i at M noaSejung YangUniversity of Hawai i at M noaThe assessment of language endangerment requires accurate estimates of speakerpopulations, including information about the proficiency of different groups withinthose populations. Typically, this information is based on self-assessments, amethodology whose reliability is open to question. We outline an approach thatseeks to improve the accuracy of self-assessment by exposing participants to asimple linguistic task before they render their judgments. The viability of the ap-proach is evaluated with the help of a case study involving 81 partial speakers ofJejueo, a critically endangered Koreanic Introduction1A variety of metrics have been proposed for assessing a language sdegree of endangerment the UNESCO system (2003), the Graded IntergenerationalDisruption Scale (GIDS; Fishman 1991), the Extended Graded Intergenerational Dis-ruption Scale (E-GIDS; Lewis & Simons 2010), and the Language EndangermentIndex (LEI; Lee & Van Way 2016), among others.

2 A key component of these andother systems is the need for accurate estimates, both of the number of speakers andof their fluency in the language. This raises an obvious question: where does thisinformation come from, and how dependable is it?This issue is rarely addressed, although it is recognized to be problematic ( ,Grenoble 2013: 28 and the references cited there). Organizations such as Ethnologue,UNESCO and the Catalogue of Endangered Languages2rely on two principal sources1 This work was supported by the Core University Program for Korean Studies through the Ministry ofEducation of the Republic of Korea and the Korean Studies Promotion Service of the Academy of KoreanStudies (AKS-2015-OLU-2250005). We wish to express our thanks to two anonymous reviewers for theirinsightful comments and to the many individuals in different areas of Korea who took part in our are especially grateful to Dr. Hyunah Ahn for her invaluable assistance with the statistical : , UNESCO Atlas: ,Catalogue of Endangered Languages: under Creative CommonsAttribution-NonCommercial InternationalE-ISSN 1934-5275 Toward a linguistically realistic assessment of language vitality104of information for their estimates: census data and fieldworker reports.

3 Crucially,both sources depend heavily on self-assessments by individual speakers in responseto queries about their language use and proficiency. This is potentially problematicfor two very different , individuals can differ in their views about what it means to speak or under-stand a language. Does it suffice to be able to exchange greetings and pleasantries?Is it enough to be able to engage in basic every-day conversations that make few de-mands on the language s lexical and morphosyntactic resources? What about fluencyin a mixed variety that is substantially different from the traditional language butgoes by the same name? These and other factors could easily shape self-assessments ofproficiency in unexpected and undesirable ways, ultimately contributing to a skewedpicture of a language s second problem is noted by Grenoble (2013). In responding to queries aboutproficiency, speakers could be swayed by extraneous factors involving their attitudeand that of others to the language in question.

4 On the one hand, speakers might betempted to overstate their proficiency in order to emphasize their sense of belongingto a particular ethnic group, consistent with the well-known fact that language is amajor marker of identity and group membership. As Grenoble notes (2013: 29): Be-cause language is an integral part of identity, people who identify with a particularethnolinguistic (or heritage) culture may claim knowledge of the language even whenthey are far from fluent. On the other hand, people may sometimes choose tounder-statetheir proficiency in order to minimize their association with a linguistic or ethnicgroup that is traditionally the victim of prejudice and discrimination. When peopleare repressed for their ethnicity, Grenoble notes, they may claim not to know thatparticular language for fear of retribution. We return to these points in Section over the accuracy of estimates of language proficiency has been expressedin other quarters as well.

5 In its 2016 report on the language data in the Cen-sus, the Commission on Language Learning of the American Academy of Arts andSciences (AAAS) urges further study of whether people who reported proficiency inparticular languages can actually use them effectively (AAAS 2016: 20). The Com-mission underlines its concern by noting that research from outside the federal gov-ernment suggests that the most recent census overestimated the number of bilingualsin the by a factor of two (AAAS 2016: 4).We outline here a methodology that seeks to take a first step toward the elicitationof self-assessments that more accurately reflect a speaker s actual level of linguisticproficiency. We begin, in the next section, by outlining the technique and illustratingits application to Jejueo, a critically endangered Koreanic language. Section 3 offersevidence that the method is more accurate than a simple request for a self-assessmentof proficiency.

6 We present some general concluding remarks in Section The self-assessment of proficiency in JejueoJejueo is the traditional languageof Jeju Island, a province of Korea located about 45 miles south of the Korean main-land. According to theCatalogue of Endangered Languages(2016), there are be-tween 5,000 and 10,000 speakers of varying proficiency on Jeju Island, as well as (toLanguageDocumentation& ,2017 Toward a linguistically realistic assessment of language vitality105a lesser extent) on the Korean mainland, in Japan, and in a few other countries aspart of the Korean diaspora. Fully fluent speakers are for the most part elderly (intheir 70s and 80s), and children are no longer learning the language. Adults aged 20to 60 show varying degrees of partial initially worked with a total of 65 participants, aged 20 to 29, all life-longresidents of Jeju Island and all fully fluent and literate in Korean, their dominantlanguage.

7 We chose participants in this age range because they are part of the pivotalgeneration of Jeju Islanders on whose shoulders the perilous future of the began by conducting a traditional language survey that included a requestto the participants to rate their comprehension ability on a five-point scale, with 1representing low ability and 5 indicating high proficiency. The average rating, acrossall participants, was , indicating an intermediate level of , we had each participant listen to 10 recorded Jejueo sentences, includingthe five sample items that appear in translated form below. A complete list can befound in the appendix, along with a list of sample Korean A small bird holding a big pumpkin seed in its mouth was flying You break this long stick into two If the weather is fine, climb up a tree, pick fruit and throw it to the Because s/he lives near her/his mother, s/he doesn t know how large the Father is patting his back with a thin stick to knock off the stuck sentences instantiated a set of monoclausal and biclausal constructions of varyingcomplexity and frequency of usage.

8 However, in order to ensure that the sentenceswere not too difficult and would not overburden the working memory of partici-pants, the test sentences were composed of basic vocabulary items, mostly from theSwadesh list, inflected and arranged in ways typical of Jejueo speech. Grammaticalmarkers (nominal and verbal suffixes and connectives) were drawn from the inven-tory of markers discussed in the authors forthcoming grammar of Jejueo; most alsoappear in the authors forthcoming four-volume Jejueo textbook, which is intendedfor use in first- and second-year courses in the language and therefore focuses on themost basic features of Jejueo sentence was pronounced twice in the recording once by a malespeaker and once by a female speaker, both of whom were in their seventies and highlyfluent. In accordance with the experimenter s instructions, participants paraphrasedeach sentence in written Korean (their dominant language) right after hearing it atask that draws heavily on their comprehension skills.

9 After completing all ten testitems, they were once again asked to rate their ability to understand Jejueo on a scaleof 1 (low) to 5 (high). On average, self-assessments fell by points, from a pre-testmean of to The difference between the two scores was highly significant(t ( ) = , p < .0001).LanguageDocumentation& ,2017 Toward a linguistically realistic assessment of language vitality106 Table scores before and after the comprehension testBefore After Mean the 65 participants, 43 adjusted their proficiency estimate downward; only threeexhibited the reverse results suggest that many participants initially over-estimated their ability tounderstand Jejueo, but that they were able and willing to adjust their self-assessmentonce they realized their mistake, thanks to the comprehension task. But a crucial ques-tion now arises: can we be sure that the right people lowered their self-assessment?The answer to this question depends on whether there is a relationship between perfor-mance on the comprehension task, on the one hand, and the likelihood of a loweredself-assessment, on the other.

10 We address this issue in the next ReliabilityIn order to get at the question of whether participants performanceon the translation task directly contributed to their revised self-assessment, we firsthad to develop a measure of their comprehension success. We did this by analyzingtheir Korean translations of the Jejueo sentences and assigning one point for eachnoun, verb, and verbal suffix that was correctly rendered, thereby arriving at a per-centage correct score for each participant. This was a relatively straightforwardmatter for the two languages that are the focus of our study. As can be seen by com-paring the sample Korean translations with the Jejueo test items in the appendix, thetwo languages are closely related, with similar word order and verbal structure (notunlike, say, Italian and Spanish or Norwegian and Dutch). As the following exam-ple helps illustrate, these parallels made it quite easy to identify missing or incorrectnouns, verbs and suffixes in the Korean translations.