Journal Impact Factors and Self-Citations: Implications ...

1 Agree that it is a misinterpretation of theSteele and Aronson (1995) results to con-clude that eliminating stereotype threateliminates the African American Whitetest-score gap. They agree that we have iden-tified multiple mischaracterizations of theirwork in media reports, Journal articles, andtextbooks, which wrongly interpret theirwork asfinding that eliminating stereotypethreat did indeed eliminate the score agree that these mischaracterizationsare , Steele and Aronson (2004)assert that there is no need to worry aboutmischaracterizations of theirfindings in theabsence of evidence that these mischaracter-izations have led to widespread misunder-standing of the role stereotype threat plays inexplaining the African American White test-score gap. We disagree. Although evidenceof such misunderstanding would certainly begrave cause for concern, we believe it issufficiently worrisome when one of the sem-inal studies on stereotype threat is commonlywrongly interpreted by the popular media,textbook publishers, and academics alike tomean that the African American White test-score gap disappears when stereotype threatis eliminated.

2 Steele and Aronson assert thattheir 1995 study is a drop in an ocean ofinformation about the race gap (Steele &Aronson, 2004, p. 47). We believe they areunduly modest about the Impact of their pa-per; that the Social Sciences citation Indexreports that it has been cited more than 300times is one indicator of its and Aronson (2004) assert thatbecause there are now over 100 researchstudies on stereotype threat, our focus onthefirst article on the topic results in aserious bias. However, they later acknowl-edge that their article is one of few stereo-type threat studies focusing on AfricanAmericans. As the African American White score gap was the topic of our arti-cle, we see our focus on this pivotal andhighly cited article as entirely and Aronson (2004) also assertthat no attentive reader of the literature onthe race gap would conclude that stereo-type threat is its sole cause.

3 However, ourconcern is with broader audiences than theserious scholar working on issues of are concerned about students who arebeing initially exposed to issues of psycho-logical testing and the race gap in theirintroductory psychology courses. We areconcerned about managers responsible forpersonnel selection systems in their orga-nizations. We are concerned about psy-chologists who do not follow testing issuesclosely and whose only exposure to stereo-type threat may be through an AmericanPsychological AssociationMonitor on Psy-chologycolumn making the interpretive er-ror that is the focus of our article. We areconcerned about the large audience watch-ingFrontlineand hearing that the score gapis eliminated in the no-threat and Aronson (2004) address theuse of a prior SAT score as a covariate,claiming that we overworried about readersbeing misled by this analysis. They argue thata larger literature shows the stereotype threateffect, sometimes with the use of a prior testas a covariate and sometimes without.

4 How-ever, in our article, we noted clearly that weare not questioning thefinding of a stereotypethreat effect ( , thefinding of a Race Diagnostic Condition interaction) in Steeleand Aronson (1995). Our concern is withmisinterpreting the graphical presentation offindings as suggesting that group differencescan be and Aronson (2004) take issuewith our comparison of African American White differences on the prior SAT and onGRE-based scores in the two experimentalconditions. Steele and Aronson assert thatthese are not comparable because the pretestSAT and the experimental GRE-based testare not perfectly correlated and becauseNissmall. Given the extensive data on the simi-larity of the score gaps between the two testsand the correlation between the two, we see itas reasonable to posit that two groups that donot differ on the SAT would also be expectednot to differ on the share with Steele and Aronson thebeliefs that single experiments do not an-swer all questions and that it is important toexamine the role of stereotype threat inreal-life testing settings.

5 We certainly agreewith their position that evolving literatureshave self -correcting capacities, and weview our article as fulfilling exactly such arole. Most crucially, we note that the dis-agreement between us is about the conse-quences of the mischaracterization we doc-umented, not about whether the work hasbeen , P. R., Hardison, C. M., & Cullen, M. J.(2004). On interpreting stereotype threat asaccounting for African American White dif-ferences on cognitive Psy-chologist, 59,7 , C. M., & Aronson, J. (1995). Stereotypethreat and the intellectual test performance ofAfrican of Personalityand Social Psychology, 69,797 , C. M., & Aronson, J. (2004). Stereotypethreat does not live by Steele and Aronson(1995) Psychologist, 59,47 be addressed to Paul R. Sackett, Depart-ment of Psychology, University of Minnesota,Elliott Hall, 75 E. River Road, Minneapolis, MN55455.

6 E-mail: Impact Factors andSelf-Citations: Implications forPsychology JournalsFrederik Anseel, Wouter Duyck, andWouter De BaeneGhent UniversityMarc BrysbaertRoyal Holloway University of LondonRecently, Adair and Vohra (January 2003)analyzed changes in the number of refer-ences and citations in psychology journalsas a consequence of the current knowledgeexplosion. In their study, the authors madea striking observation of the sometimes ex-cessive number of self -citations in psychol-ogy journals. However, after this illustra-tion, no further attention was paid to theissue of self - citation . This is unfortunatebecause little is known about self -citingpractices in psychology. Early research onself-citations in psychology journals indi-cated that about 10% of citations were self -citations, and one author concluded that itis apparent that controlling for self -citationis not necessary (Gottfredson, 1978, ).

7 Similarly, although thePublicationManual of the American Psychological As-sociation(American Psychological Associ-ation, 2001) provides clear guidelines onthe form citations should take, it does notindicate when it is appropriate to cite one sown urge more cautionwhen dealing with self -citations. A multi-disciplinary study found that 36% of allcitations represent author self -citations(Aksnes, 2003; see also McGarty, 2000, fora similarfinding in social psychology). Es-pecially troublesome is thefinding thatself-citations peak during thefirst threeyears after publication, thereby strongly in-fluencing Impact Factors of journals that arebased on two-year the use of citation counts(and Impact Factors ) has been criticized inall disciplines (see, , Boor, 1982), it hasbecome the main quantitative measure ofthe quality of a Journal . Accordingly, thesefigures are used to make decisions about49 January 2004 American Psychologistjournal subscriptions, selection of journalsfor paper submission, rankings of authors,author tenure, individual grants, and evenfunding of entire research groups and insti-tutions, insofar that researchers have ex-pressed their concern about the current ob-session of the academic world with impactfactors (Lawrence, 2003).

8 Therefore, an important underex-plored question is to what extent impactfactors of psychology journals are artifi-cially inflated or deflated by are good reasons to expect moreself-citations from authors publishing inhigh- Impact journals ( ,PsychologicalBulletin) than from authors publishing inlow- Impact journals ( ,Computers inHuman Behavior). This is because theformer authors in general are more experi-enced and more successful. In addition,they are often part of a larger researchgroup, so that they can coauthor more ar-ticles. For example, researchers who pub-lished inPsychological Bulletinduring1998 1999 had on average publica-tions in 2000, in comparison with for authors who published inComputers in Human a con-sequence, these researchers have more op-portunities to cite their own work. Themain question, however, is whether thisdifference in self - citation rate is strongenough to affect the Journal Impact scores:Do journals with high citation counts get anextra citation boost due to self -citations?

9 We used the Institute for ScientificInformation (ISI) databases Web of Sci-ence (WoS) and Journal citation Reports(JCR) as the basis of our analyses. Fromeach article (including empirical articlesand literature reviews) infive high-,fivemiddle-, andfive low-ranked journals inpsychology published in 1998 and 1999,we collected the number of self -citationsand other-citations in 2000 from the journals were included in either thepsychologyorpsychology, experimentalsubsection of the JCR Social Science Edi-tion 2000. self -citations were assessed us-ing the criterion that at least one author(first or coauthor) was also an author (firstor coauthor) of the citing paper. On thebasis of these data, we recalculated theimpact factor for each Journal with andwithout self -citations. As can be seen inTable 1, small discrepancies between theofficial Impact Factors (ISI) and our owncalculations appeared.

10 These discrepanciesare caused by inaccuracies in the ISI data-bases and have been reported before ( Er-rors in citation Statistics, 2002).We found that articles in high-impactjournals received more (M ) self -citations than articles in middle- (M ) and low- Impact journals (M ),thus confirming our expectation that au-thors publishing in high- Impact journalshave more opportunities to cite their ownwork. To test whether these differences innumbers of self -citations change the differ-ences in Journal Impact Factors , we calcu-lated the ratio of self -citations to the totalnumber of citations for each article thatreceived citations in 2000. A one-way anal-ysis of variance was calculated with thisratio as a dependent variable and impactfactor group (high, middle, low) as the in-dependent variable. There was a significanteffect of Impact factor group,F(2, 650) ,p .001, 2.