Evaluating counseling process and client outcomes.

1 VISTAS Online is an innovative publication produced for the American counseling Association by Dr. Garry R. Walz and Dr. Jeanne C. Bleuer of counseling Outfitters, LLC. Its purpose is to provide a means of capturing the ideas, information and experiences generated by the annual ACA Conference and selected ACA Division Conferences. Papers on a program or practice that has been validated through research or experience may also be submitted. This digital collection of peer-reviewed articles is authored by counselors, for counselors. VISTAS Online contains the full text of over 500 proprietary counseling articles published from 2004 to present. VISTAS articles and ACA Digests are located in the ACA Online Library. To access the ACA Online Library, go to and scroll down to the LIBRARY tab on the left of the Under the Start Your Search Now box, you may search by author, title and key The ACA Online Library is a member s only benefit. You can join today via the web: and via the phone: 800-347-6647 is commissioned by and is property of the American counseling Association, 5999 Stevenson Avenue, Alexandria, VA 22304.

2 No part of Vistas may be reproduced without express permission of the American counseling Association. All rights ACA at: VISTAS Online229 Article22 EvaluatingCounselingProcessandClientOutc omesMarloweH. Smaby,Cleborne D. Maddux,IreonLeBeauf, and Jill PackmanRec entl y, th ere has bee n inc rea se d em pha sis on obje ctivemea suresof the quali ty of coun sel ing , part icu lar ly fo r use in thetr ainingof com mon topic,no t only in counselingeducation,but alsoin the educ at ion of teachers,ph ysi cia ns, engi nee rs, an d othe rs(Ho wle y, 2004;Smabyet al. , 20 05; Urbani et al. , 20 02; Vaug h &Ev erhart, 2004).In any fi eld , bes t pra ct ic es in asse ssm ent ma ke use of mu lti plemea surement s of bothproc ess an d out com es. In the fiel d ofcounseling, mea surementssh oul d be ob tai ned from coun sel ors , client s and ssm en t in str umen ts shou ld mee t soun d psyc hom etr an d va lidit y data shou ld ensurethat measurementsare consi ste nt and measurewhat th ey are int end ed to mea sur (1995)suggestedthat whileself assessmentsby counselorsas wel l as cl ient ratingscalesare of ten us ed and can provideusefulinf orm at ion, many are no t of val ue beca use they are no most commonassessmentapp roachused by researchersin volv es the use of tra ined rate rs who rate a counseloron variouscounseling pro ces ses.

3 Vide o reco rdi ngs of the cou nse lor areCompelling CounselingInterventions230ben efici al to pr eser ve the perf ormancefor trainingpurposesand tofaci lit ate the est ab lishmentof inte rra ter reli abi lit y (Sm aby et al. ,20 05) . Counselin g outcomeassess men t shou ld also mak e us e oftr ai ned rater t an d Fa ust (2003) con clu de d tha t mor e res ear chshouldbe conductedto deter mineho w best to evaluatethe efficacyof graduate counselingeducationwith var ious counselingprocessesand clientoutcomesas y #1 Scha efle, Sm aby, Packman,and Maddux(2007)investigatedwhethercounselor s who had been tr ainedin the SkilledCounselorTraining Model (SCTM)co uld tran sfe r cou nse lin g ski lls tocou nselingsessions with real clientsand whethercounselingskillsare rela ted to cou nselingoutcomes. Th e mea sur e of cou nse lin gproce ss was scores on the Sk ill ed Co uns el ing Scale(SCS),a reli ablean d val id inst rume nt for measuring coun selin g pr oce ss.

4 Th e SC Smeas ures 18 spec ifi c coun sel ing skil ls and em plo ys a rank ing sy ste mused by exper t raters and other me asure of coun seli ng out com e wa s al so incl ude d. TheGoal Attai nmentScale(GAS;Kir esuk,Smith,& Cardillo,1994)isa rel iableand validinstrumentthat is completedby counselorsandis a met hod for constructingtai lor -ma de co uns eli ng go als anddet erm iningwhe ther or not they hav e been e Co unsel or ResponseFor m - Sh ort (CRS-F;Tracy,Gli dden,& Koko to vic, 19 88) wa s us ed and wa s comp let ed by theclient s. Th is rel ia ble and vali d inst rum ent assessesclients per ceptionsof co unse lo rs soci al inf luence(at tr activeness,expertness,andtr ust wor th in es s) .Pa rticipants in thi s stud y wer e cou nse lin g maste r s stude nts ata large, LandGr ant Uni versityin the who were enrolledin a gr ad uat e in ter ns hip. The pr og ram is acc reditedby the CouncilforAc cr edi tat ion of Counselingand Rela ted Edu cat ion al Prog ram sEval uati ng Counseli ng Pr ocessand Cli ent Out comes231(CA CREP).

5 Sk il led counseling Sca le (SCS ) ra tin gs, goa l at tai nme ntsc ores (GAS),and soc ial influ en ce scor es (CRF-S) wer e all ve ry hi to thr ee goa ls were id ent ifi ed by th e cou nse lor s for the GA Sins trument. The percen tage who reportedthat they thoughttheyac hie ve d thes e goa ls were 91% , 91%, and 87 %, respectively. Almostal l the pa rtic ipa ti ng coun sel ors re ported tha t they belie ved that almostall th e clien ts they ha d wo rked wit h had achieved alm ost all th eir goal F- S scores were als o very high . Th e med ian scor e wa s 78,wi th 84 bein g th e hig hest possi ble sc ore. An analysisof individualit ems sh ow ed tha t the mediansc ore was on all it ems (the high estpos si bl e sco re wa s ). Thesefi nd ings showtha t th e cl ien ts of th esecou nsel ors belie ved theyhad hi gh levelsof socialinfluence(at tr act iven es s, exp ertness,an d trustworthi nes s).

6 Th e res earc hers co nclu ded th at the co uns elo rs in thi s stud ywereabl e to transfer the counseling skillsthey learnedto actualsessionswi th ,counselors reportedhigh levelsof goalat tai nmen t and cl ien ts perc eivedhig h le vel s of soc ial influe nce in the irco uns elo rs .Study#2 Mall inckr odt and Nelson(1991)focusedtheir researchon acounseling var iablecalledthe wor king alliancebetweencounselorand cli ent. The wor king allianceis definedas bonding,goal setting,and task ide nti fic Wor kin g All ian ce Invent ory (WAI;Ho rva th & Sy mon ds, 19 91) was us ed to measurethis vari able and tost udy the re la ti onsh ip wit h leve l of cou WAI hasbeen shownto be a reliableand valid measureof counselingprocessand rt ic ipant s included50 counselo rs and 76 novices(completedfirst semesterof cou nseloredu cat ion pro gr am), 24 advancedtrainees(completedcounselingin te rnsh ip s) , an d 6 experienced coun sel ors (p ost do cto ral st ude nts ).

7 Le vel of tra in ing wa s not fou nd to be relatedto bondingin thecl ient -c ou nse lor re lationship, bu t th e experi enc ed coun sel ors wer efoundto have betterscor es on goal-settingand task identificationthan the novi ce or advancedtr ain ees . The nov ice an d advan cedCompelling CounselingInterventions232tra inee s appe ar ed to focusexclu siv ely on bo ndi ng and negl ect edgo al -se tti ng and identifyingta sks aim ed at re ach ing thos e goa perienced counselorswere adeptat bonding,goal-setting,and taskiden ti fi ca tio n. Th e rese arc her s con clu ded that traineessho uld learn touse bondingin com binationwit h goa l set tin g and de lin eat ing sp eci ficta sks for achieving e re sea rc h sum marizedabo ve re infor ces th e reco mme nda tio nthat co uns elo rs use multiple as ses smen t inst rum ent s and tha t thes eins tr ume nts sho uld be basedupon soundpsychometricprinciples.

8 Withthes e fin ding s in mind, the authorsha ve id ent ifi ed two met hod s fo rev alua ti ng co uns eling pr ocessand out come s. Method#1 includesuseof on e instr ument for assessingcoun seli ng pro ces s (SCS ) and twoinst rum en ts for ass essingcounseli ng outc ome s (G AS and CRF- S).Meth od #2 in clu de s one instrumentfo r asse ssi ng bo th co uns eli ngpr oce ss and outc omes(WA I).Met hod #1 is us eful when spe cif ic cou nse lin g skil ls areta ught usinga systematicskillstr ainingprogramsuch as the SCTM,Mic rocounseling, or #1 isals o us ef ul whe n the counselor ide nti fies concretegoalsand tasksand wha t is des ir ed is a met hod of eva luationbasedon n too, Me thod #1 can be use d to mea su re perc ept ion s of howinf lu en tial the counseloris in termsof attractive nes s, exp ertne ss, an dtr ustw ort #2 is usefulto assessoverallqualityof thecounselingrelationship,the abilityto set co uns elinggoals an d theabi li ty to identi fy tasksaimedat #2 isan ass es smen t pro ce dur e that is more gl obal in naturethan is Method#1.

9 Thesemethod s are descr ibed thod#1Th is meth od mak es use of thr ee inst ruments:the SCS,theGAS,and the CR F- S. The SCS inc lud es 18 it ems rated on a five-point,Li ke rt -ty pe sca le. Th e itemsare or gani zed int o three stages(exploring,Eval uati ng Counseli ng Pr ocessand Cli ent Out comes233understanding, and acting). Eachstageis madeup of two processes,each wit h three exploringstageincludesthe processesof attending,andquestioningand reflecting. Ex pl or in g sta ge skil ls in th e at ten din gprocessar e maintainingeye contact,usingbody la nguage,an d verbalfollo wing . Exp lo ri ng stag e skil ls in th e questioningand refl ect ingproc ess incl ude open-ended ques tio ni ng, para phr asi ng, andsummarizi der sta ndi ng stageinc lud es the proc ess es ofin terc han gea ble em pat hy and addi tive empat hy . Understandingstageskill s in the interchangeableempathypr ocessare statingfeelingandconte nt, sel f-d isc losure,and ask ing for co ncr ete and spe cif icexpr essi ons.

10 Un der standingsta ge skil ls in th e addi tive emp ath yproc ess are imm ediacy; ident ify ing gen eral problem,actiontaken,and feeli ngs; and conf rontingin a car ing way .The actingstageinc ludesthe pr ocessesof decision-makingand cont ract ing. Actingstageskillsin the decision-makingprocessare deciding, choosing,and lls in the co nt ra ct ing process inc lud e reachingagreements,settingdeadlines,and re viewinggoalsand actionsto instrumentis av ailablein an in pre ss bo ok by the au tho rs(Smaby& Maddux, in press ). A fu ll des cr ipt ion of the inst rum ent can befo un d in th at book and in Schaefle,Smaby,Mad dux , an d Ca tes (2 005 ).The SCS can be filledout by the counseloras a form of self-assessment , and /o r by expert,tr ain ed rater s. Urba ni et al . (2 002 )re porte d a mean self -rat ing scor e for th a st andarddeviationof ,and meanratingby expert,tra ined rate rs of with a sta nda rd devia tio n of 2.