Speech Recognition Testing: The Basics

1 Speech Recognition testing : The BasicsRachel McArdle, Chief, Audiology and Speech Pathology ServiceResearch Career Development Awardee, VA RR&DDepartment of Veterans AffairsBay Pines VA Healthcare System, Bay Pines, FLAssociate Professor, Communication Sciences & DisordersUniversity of South Florida, Tampa, FLBay Pines VA Healthcare System AcknowledgementsThis material is based upon work supported by the Department of Veterans Affairs, Veterans Health Administration, and Office of Research and Development, Rehabilitation Research and Development contents of this presentation do not represent the views of the Department of Veterans Affairs or the United States Components of Hearing Loss Carhart (1951) Loss of acuity Loss of clarity Stephens (1976) Simple attenuation Major distortions Plomp (1978) Audibility Distortion NormalImpairedAudibilityDistortionBackgr oundNoiseSpeechSpeechSpeechSpeechSpeechS peechSpeechSpeechSPEECH PERCEPTION EFFECTS OFSENSORINEURAL IMPAIRMENTSPEECH PERCEPTION EFFECTS OFSENSORINEURAL IMPAIRMENTB oothroydAudibility measures of Speech Speech Recognition threshold (SRT) Purpose ASHA method Recorded materialsAudibility measures of Speech Speech Recognition threshold (SRT) Speech Recognition in quiet Phonetically-balanced lists-4-20246 WORD LISTS50% POINT (dB S/N)PB-50W-22NU No.

2 6 RANDOMLY SELECTEDThe mean 50% points for the individual words calculated with theSpearman-K rber equation are shown for lists 1-4 of the PB-50 (triangles), of the CID W-22 (squares), and of the NU No. 6 (inverted triangles). The 12 randomly compiled lists (circles) are shown in the right half of the figure. The vertical bars indicate one standard deviation. Wilson, McArdle, & Roberts, JAAA, 2008 Audibility measures of Speech Speech Recognition threshold (SRT) Speech Recognition in quiet Phonetically-balanced lists MLV vsrecorded materialsRoeser & Clark, AT, 2008 Audibility measures of Speech Speech Recognition threshold (SRT) Speech Recognition in quiet Phonetically-balanced lists MLV vsrecorded materials Speaker differencesAudibility measures of Speech Speech Recognition threshold (SRT) Speech Recognition in quiet Phonetically-balanced lists MLV vsrecorded materials Speaker differences List differences (NU 6, W-22, PB-50) PB-50 harder than the W-22 (Hirsh et al, 1952) NU 6 harder than W-22 020406080100020406080100 CORRECT Recognition (%)-7-238-7-238 SIGNAL-TO-NOISE RATIO (dB)PB-50 LIST 8 LIST 9 LIST 10 LIST 11W-22 LIST 1 LIST 2 LIST 3 LIST 4NU No.

3 6 LIST 1 LIST 2 LIST 3 LIST 4W-22NU No. 6PB-50 MeansSpeaker Differences!The mean percent correct Recognition at four signal-to-noise ratios for the four lists of each of the three monosyllabic words list materials. The mean data for each of the lists are illustrated in the 4th quadrant. The lines with each set of data are the linear regressions used to describe the data. The 50% point for each function and the slope of the function at the 50% point arelisted in Table 2. The horizontal line in each panel indicates the 50% point on each , McArdle, & Roberts, JAAA, 2008 Speech -in-noise testing Came about in the late 1960 s as a way to quantify the amount of distortion Carhart& Tillman (1970) advocated for Speech -in-noise testing to be part of test battery Strom (2006) surveyed and found that less than half of dispensing professionals use some type of Speech -in-noise taskSNR loss(Killion, Seminars in Hearing, 2002)Predicting Speech Recognition Performance in Noise Audibility Linear Easily predicted from pure tones ( , AI)COCHLEAR#1#2#3020406080100 PERCENT CORRECT Recognition #4#5 ROLL OVERRETROCOCHLEAR NORMAL CONDUCTIVE 406080100#1 PRESENTATION LEVEL (dB HL)% Correct Recognition PerformancePredicting Speech Recognition Performance in Noise Audibility Linear Easily predicted from pure tones ( , AI)

4 Distortion Non-linear Poor prediction from pure tones0102030405060700481216202450% Point WORDS-IN-NOISE (dB S/B)PURE-TONE AVERAGE (dB HL)500, 1000, 2000 Hz1000, 2000, 4000 Hz010203040506070N = 315r = = Speech Recognition Performance in Noise Audibility Linear Easily predicted from pure tones ( , AI) Distortion Non-linear Poor prediction from pure tones Poor prediction from word Recognition performance in quiet02040608010004812162024% CORRECT Recognition at 80-dB HL50% CORRECT POINT (dB S/B)176 ( )107 ( )104 ( )Wilson & McArdle, Journal of Rehabilitation Research and Development, 2005 Predicting Speech Recognition Performance in Noise Audibility Linear Easily predicted from pure tones ( , AI) Distortion Non-linear Poor prediction from pure tones Poor prediction from word Recognition performance in quietSpeech Recognition in noise performance must be measured directly!0246810121416182021231246810121 41618 QUICKSIN LIST50% CORRECT POINT (dB S/N)Listeners with Normal HearingMcArdle & Wilson, Journal of the American Academy of Audiology, 2006024681012141618202123124681012141618 QUICKSIN LIST50% CORRECT POINT (dB S/N)McArdle & Wilson, Journal of the American Academy of Audiology, 2006 Listeners with Hearing LossHINHMcArdle & Wilson, Journal of the American Academy of Audiology, 2006 PURE-TONE AVERAGE (dB HL) (500, 1000, 2000, & 4000 Hz)20304050607080 AGE IN YEARS50% CORRECT POINT (dB S/B)0102030405060-404812162024 Function of Aging?

5 Wilson & Weakley, Journal of the American Academy of Audiology, 2005 Why measure Speech -in-noise in an audiologic evaluation? Addresses common complaint of the patient Difficulty understand Speech in background noise The data provide insight into the most appropriate amplification strategy Directional microphones, FM systems Counseling, realistic expectationsSpeech-in-Noise Tests Sentence tests SPIN HINT QuickSIN BKB-SIN Monosyllable tests WIN SPRINTS peech Perception in Noise Test (SPIN) The amount of semantic context leading to the last word of each sentence, which is a monosyllabic target word, is varied 50 sentences (25 LP, 25 HP) scored as the percentage of LP and HP words correctly perceived Examples:Low Predictability (LP)Ruth s grandmother discussed the broomHigh Predictability (HP)The girl swept the floor with a broomQuietNoise +6 dB S/NKalikow et al, Journal of the Acoustical Society of America, 1977 TimePrePost6-mo1-yrProbability Correct406080100HA-alone HP HA-alone LP HA+AR HP HA+AR LP HighPredictabilityN = 105 LowPredictabilityHearing in Noise Test (HINT) 10 BKB sentences-1stgrade reading level Repeat entire sentence correctly bracketing method Decrease signal for correct answer Increase signal for incorrect answer Speech spectrum noise -fixed Scored in terms of signal-to-noise ratio at the 50% pointExample.

6 Her shoes were very dirtyQuiet Noise (3 dB S/N)Nilsson et al, Journal of the Acoustical Society of America, 1994 HINTN oise = 70 dB HL#List 1 - practiceLevelList 1 Level1223344556677889910(A/The) boy fell from (a/the) window(A/The) boy fell from (a/the) window(A/The) boy fell from (a/the) window(A/The) boy fell from (a/the) window(A/The) wife helped her husband(A/The) boy fell from (a/the) window(A/The) boy fell from (a/the) window(A/The) boy fell from (a/the) window(A/The) boy fell from (a/the) window(A/The) boy fell from (a/the) windowBig dogs can be dangerousHer shoes (are/were) very dirty(A/The) player lost (a/the) shoeSomebody stole the money(A/The) fire (is/was) very hotShe's drinking from her own cup(A/The) picture came from (a/the) book(A/The) car (is/was) going too fast84+82+84+82+80+82+84/7 = (signal) 70 dB HL (noise)50% point = dB S/NBKB Speech -in-Noise Test (BKB-SIN) 10 BKB sentences-1stgrade reading level 3 target words per sentence Multitalker babble Descending paradigm +21 to -6 dB S/N 3 dB decrements Scored in terms of signal-to-noise ratio at the 50% point Good for CI users, children, and profound hearing lossExample:The bagfellto the groundQuiet Noise (3 dB S/N)EtymoticResearch, 2005 BKB-SIN Test List mouserandownthe hole.

7 4+ lightwentout3+ potatoes3+ littlegirlis shouting3+ coldmilkis in a pitcher3+ paintdrippedon the ground3+ tea3+ fatheris cominghome30 her spendingmoney. bringinghis raincoat3-6xxxxxxxxxxxxx433332100019 SNR 50%= 19 = dBSpearman-K rber Equation(Finney, 1952) 50% = i + (d) (d)(# correct)/(w) i= the initial presentation level (dB S/B) d= the attenuation step size (decrement) w= the number of items per decrement. BKB Example:Initial starting level 21 dB S/N, 3 dB step size, 3 words per decrement # correct = 1550% = 21 + (3) (3)(15)/350% = 21 + -1550% = +1* 15 50% = 15 = dB S/NQuick Speech -in-Noise Test (QuickSIN) 6 IEEE sentences- 5 target words per sentence Syntactic cues but subtle semantic cues Multitalker babble Descending paradigm 25-to 0-dB S/N 5 dB decrements Scored in terms of signal-to-noise ratio at the 50% point (Spearman-K rber equation)Quiet Noise (5 dB S/N)Example:It is a bandof steel3incheswideKillion et al, Journal of the Acoustical Society of America, 2004 QuickSIN List 1A whitesilkjacketgoes with any shoe.

8 S/N 25 The childcrawledintothe densegrass. S/N 20 Footprintsshowedthe pathhe tookup the 15 A ventnearthe edgebrought in 10 It is a band of steel three inches 5 The weightof the packagewasseen on the high scale. S/N Total Correct = SNR LossxxxxxxSCORE554330 QuickSIN Example:Initial starting level 25 dB S/N, 5 dB step size, 5 words per decrement # correct = 2050% = 25 + (5) (5)(20)/550% = 25 + -2050% = 20 = dB S/NSNR loss 50% = 20 = dB S/N0102030405060708090100 CORRECT WORD Recognition (%)252123124681012141618 QUICKSIN LIST5101520 Stimulus variabilityDimes showered down from all sidesIt was done before the boy could see itn = 72 listeners with hearing lossMcArdle & Wilson, Journal of the American Academy of Audiology, 2006-4-3-2-101234124681012141618 QUICKSIN LISTDEVIATION FROM MEAN 50% POINT (dB)McArdle & Wilson, Journal of the American Academy of Audiology, 2006-4-3-2-101234126810121617 QUICKSIN LISTDEVIATION FROM MEAN 50% POINT (dB)11 McArdle & Wilson, Journal of the American Academy of Audiology, 2006 Words-in-Noise Test (WIN) 35 NU No.

9 6 monosyllabic words (female speaker) 5 words per signal-to-noise ratio Multitalker babble -fixed Descending paradigm 24-to 0-dB S/N 4-dB decrements Scored in terms of signal-to-noise ratio at the 50% point (Spearman-K rber equation)Example:Say the word voiceQuiet Noise (12 dB S/N)Wilson, Journal of the American Academy of Audiology, 2003 List 1, Random 2 24-dB S/B 12-dB S/B 0-dB S/B 1 FOOD 16 RUSH 31 BATH 2 PAIN 17 VOICE 32 DAB 3 LATE 18 TOOL 33 GET 4 DODGE 19 SEARCH 34 READ 5 COOL 20 GOOD 35 LIFE 20-dB S/B 8-dB S/B 6 DITCH 21 MAKE # Correct 7 KICK 22 SOAP 8 LUCK 23 YOUNG 9 GUN 24 SOUR 10 SUCH 25 HALF 16-dB S/B 4-dB S/B 11 WIRE 26 SHEEP 12 TIME 27 MESS 13 HAVE 28 MOOD 14 JUDGE 29 LONG 15 DOG 30 FAR xxxxxxxx17 WIN Example:Initial starting level 24 dB S/N, 4 dB step size, 5 words per decrement # correct = 1750% = 24 + (4) (4)(17)/550% = 24 + 2 17( )*50% = 26 = dB S/N*The " " is the attenuation step size (4 dB) divided by the number of words per step (5)

10 Name_____SS#_____Age_____ Date_____By_____Ear_____Level_____ 24-dB S/B12-dB S/B 0-dB S/B1pain 16hate 31gaze 2youth 17shack 32life 3wheat 18tool 33get 4dodge 19voice 34read 5cool 20rush 35bath 20-dB S/B 8-dB S/B 6ditch 21turn # Correct7ring 22young 8kick 23bite 9chair 24pick 10luck 25half 16-dB S/B 4-dB S/B 11base 26far 12wire 27learn 13red 28mood 14time 29talk 15judge 30note Threshold (50%) dB S/BTrack 25, List 1, Random 1 Ear_____Level_____ 24-dB S/B12-dB S/B 0-dB S/B1food 16good 31back 2road 17search 32dab 3juice 18pass 33kill 4late 19witch 34nice 5hire 20chief 35calm 20-dB S/B 8-dB S/B 6tire 21sour # Correct7such 22doll 8shawl 23deep 9haze 24soap 10gun 25make 16-dB S/B 4-dB S/B 11live 26beg 12date 27mess 13gas 28long 14have 29mouse 15dog 30sheep Threshold (50%)