Associative Learning Improves Visual Working Memory ...

1 Associative Learning Improves Visual Working Memory PerformanceIngrid R. OlsonUniversity of PennsylvaniaYuhong JiangHarvard UniversityKatherine Sledge MooreUniversity of PennsylvaniaThe ability to remember Visual stimuli over a short delay period is limited by the small capacity of visualworking Memory (VWM). Here the authors investigate the role of Learning in enhancing saw 2 spatial arrays separated by a 1-s interval. The 2 arrays were identical except for 1location. Participants had to detect the difference. Unknown to the participants, some spatial arrays wouldrepeat once every dozen trials or so for up to 32 repetitions. Spatial VWM performance increasedsignificantly when the same location changed across display repetitions, but not at all when differentlocations changed from one display repetition to another. The authors suggest that a major role oflearning in VWM is to mediate which information gets retained, rather than to directly increase : Visual short-term Memory , Learning , training, change detection, chunkingThe human cognitive system is stunningly powerful in somerespects yet surprisingly limited in others.

2 As human beings, wecan recognize an object (Thorpe, Fixe, & Marlot, 1996) or a face(Liu, Harris, & Kanwisher, 2002) in a single glimpse and type 70words per minute, yet we cannot hold more than four objects at atime in Visual Working Memory (VWM) (Luck & Vogel, 1997) orsplit our attention to several locations (Pylyshyn & Storm, 1988).Attention and Working Memory impose major capacity limitationsin cognitive processing. This study is concerned with how learningaffects the ability to retain information in of VWMVWM1allows us to hold a Visual display in mind for a fewseconds after its disappearance (Phillips, 1974). A briefly pre-sented array of items is first held in an iconic Memory , in a largelyretina-based format, for approximately 100 to 200 ms (Neisser,1967; Phillips, 1974; Sperling, 1960). Information is kept in thisformat before the icon decays, during which time a small subset istranscribed into VWM.

3 The capacity of VWM is limited to ap-proximately four Visual items (Alvarez & Cavanagh, 2004;Cowan, 2001; Luck & Vogel, 1997; Pashler, 1988; Phillips, 1974)and six spatial locations (Irwin, 1992; Jiang, Olson, & Chun,2000). These items are represented allocentrically, as a visualpattern (or configuration) encoded with reference to one another(Bor, Duncan, & Owen, 2001; Jiang et al., 2000; Phillips, 1974;Sanocki, 2003; Santa, 1977; Yantis, 1992).The small capacity approximately four items varies littlewith differences in features ( , colors, orientations, size; Vogel,Woodman, & Luck, 2001) or with familiarity ( , letters letters; Pashler, 1988). But items retained in thislimited capacity may vary in complexity according to an object-based representation (Luck & Vogel, 1997) or perceptual grouping(Lee & Chun, 2001; Woodman, Vecera, & Luck, 2003). Luck andcolleagues showed that chunking features into objects can increasethe number of features retained in VWM.

4 When each object is aconjunction of several features, four Visual objects can be heldwith the same ease and fidelity as four simple features (Luck &Vogel, 1997). However, costs become apparent when features aredrawn from the same feature category ( , color color conjunc-tions; Wheeler & Treisman, 2002), suggesting that feature heter-ogeneity of the to-be-remembered information modulates memorycapacity (Olson & Jiang, 2002; Xu, 2002). In addition to object-level chunking, multiple items can also be grouped on the basis of1 The termsvisual short-term memoryandvisual Working memoryconnote different things, withshort-term memoryemphasizing the storageaspect of Memory andworking memoryemphasizing both the storage andthe manipulation of information held in Memory . Although the distinctionbetween storage and manipulation is of theoretical interest, in practice,these terms have been used somewhat interchangeably. The change detec-tion paradigm used here, with short retention interval between two arrays,is sometimes referred to asvisual Working Memory ( , Luck & Vogel,1997) and sometimes referred to asvisual short-term Memory ( , Al-varez & Cavanagh, 2004).

5 Because there is no reason to believe that thisparadigm is only tapping into the storage and not into the manipulationaspects of immediate Memory , we have decided thatvisual Working mem-oryis a more neutral term to use for this R. Olson and Katherine Sledge Moore, Center for CognitiveNeuroscience, University of Pennsylvania; Yuhong Jiang, Department ofPsychology, Harvard thank James Brockmole, Amishi Jha, David Linden, Steve Luck,and Jon Winower for helpful comments and Sidney Burks for data collec-tion. The study was supported in part by the Clark Fund and NationalScience Foundation Grant 0345525 to Yuhong concerning this article should be addressed to Ingrid , Center for Cognitive Neuroscience, University of Pennsylvania,3720 Walnut Street, Room B51, Philadelphia, PA 19104. of Experimental Psychology:Copyright 2005 by the American Psychological AssociationHuman Perception and Performance2005, Vol.

6 31, No. 5, 889 9000096-1523/05/$ DOI: (Xu & Nakayama, 2003) or they can be remembered as asingle, complex pattern rather than as several isolated items (Jianget al., 2000).Role of LearningIn this study, we examine the effects of Learning on VWM. Weask, can humans hold more information in VWM from familiarvisual displays than from unfamiliar displays? Will they remembera display better if it is repeated over and over again? These areimportant questions because unlike laboratory stimuli, naturalscenes often remain stable over time. This affords plenty of op-portunities to learn from repeated encounters. A previous encoun-ter with a scene leaves a long-term Memory trace (Hollingworth,2004; Hollingworth & Henderson, 2002) that may affect workingmemory for that date, most studies have focused on characterizing VWM fornonrepeated Visual displays, neglecting the role of Learning . Thisforms a critical gap in the literature because Learning plays asignificant role in many other aspects of Visual cognition: thespeed of Visual search is improved by practicing the same searchtask for several sessions (Schneider & Shiffrin, 1977; Shiffrin &Schneider, 1977), dual-task interference is largely reduced afterthousands of trials of practice (Schumacher, Seymour, Glass,Kieras, & Meyer, 2001; Van Selst, Ruthruff, & Johnston, 1999),and Visual search is more efficient on familiar ( ,2s and5s) thanon unfamiliar items (rotated2s and5s; Wang, Cavanagh, & Green,1994).

7 In addition to general procedural Learning and familiarity,specific information about Visual targets can be acquired in animplicit manner. The Visual system is highly sensitive to repeatedtarget locations (Maljkovic & Nakayama, 1996), to incidentalfeatures such as target color (Maljkovic & Nakayama, 1994), tosequences of target locations and motor responses (Nissen &Bullemer, 1987; Reber, 1989; Segar, 1994), and to the associationbetween targets and distractors (Chun & Jiang, 1998, 1999, 2003;Olson & Chun, 2001a, 2001b; Olson, Chun, & Allison, 2001).General procedural Learning can enhance performance on VWMtasks, but only modestly. Olesen, Westerberg, and Klingberg(2004) gave participants an extensive amount of training for 35sessions on a spatial VWM task. Participants were required toremember several sequentially presented locations and then recallall locations after a brief retention interval. At the end of theextensive training period, accuracy improved modestly in oneexperiment and not at all in another.

8 The modest improvementseen after such an extensive amount of training can be contrastedto the large, and at times rapid, changes in Learning observed invisual search tasks (Shiffrin & Schneider, 1977) and perceptualdiscrimination tasks (Fahle & Edelman, 1993; Poggio, Fahle, &Edelman, 1992; Ramachandran & Braddick, 1973; Shiu & Pashler,1992; Vaina, Sundareswaran, & Harris, 1995).Specific practice, in the form of repeating the same exact dis-plays many times, also fails to enhance performance (Olson &Jiang, 2004). Participants in Olson and Jiang s (2004) study wererequired to remember an array of locations or shapes and to detectwhether one item had changed on a second display. Unknown toparticipants, some arrays of stimuli were repeatedly presented over20 times in the experiment, but the item that might change on thetest image was randomly chosen from the stimulus array. In otherwords, repeating a specific array did not help predict which loca-tion would be queried on the test image.

9 Surprisingly, participantsdid not benefit from the repetition of Memory displays, eventhough they recognized the repeated displays on a posttask recog-nition exam. These findings suggest that display repetition andfamiliarity is insufficient to enhance VWM together, previous studies suggest that while the visualperceptual system can gradually increase its acuity through prac-tice, VWM is relatively insensitive to practice. General practice ina VWM task produces only a small improvement in performancewith extensive training, and specific practice on repeated displaysdoes not make these displays easier to StudyThe studies outlined in this article test the hypothesis thatlearning has a limited role in enhancing how much information isstored in VWM but that it can significantly change which infor-mation is placed in VWM. This hypothesis is based on the obser-vation that past experience creates knowledge, either implicit orexplicit, about what parts of the Visual input are important andwhich parts are unimportant.

10 It is possible that the important partsof the Visual input have priority for entrance into VWM. Theprioritization hypothesis(Hypothesis 1) can be contrasted withtwo alternatives: that VWM is always impervious to Learning (Hypothesis 2:rigid VWM hypothesis) or that VWM capacityalways increases with Learning (Hypothesis 3:modifiable capacityhypothesis). Experiments 1 through 3 were designed to test thesehypotheses. A fourth experiment examined whether Learning af-fected VWM processing during the encoding or during the com-parison 1: Transfer From Nonassociative Learning toAssociative LearningIn previous experiments (Olson & Jiang, 2004) we showed thatperformance in a VWM task was not enhanced when a display wasrepeatedly encountered, even when participants could recognizethe displays at above-chance levels. In that study, the item thatmight change was randomly chosen from the Memory display, solearning the repeated display was not predictive of the potentialchange.