Automated Affect Detection in Deep Brain Stimulation for ...

1 Automated Affect Detection in Deep Brain Stimulation forObsessive- compulsive Disorder: A Pilot StudyJeffrey F. CohnUniversity of PittsburghPittsburgh, A. Jeni,Itir Onal ErtugrulCarnegie Mellon UniversityPittsburgh, MaloneCleveland ClinicCleveland , S. OkunUniversity of FloridaGainesville, BortonBrown UniversityProvidence, Rhode K. GoodmanBaylor College of MedicineHouston, measurement of affective behavior in psychopathologyhas been limited primarily to screening and diagnosis. While useful,clinicians more often are concerned with whether patients are im-proving in response to treatment. Are symptoms abating, is affectbecoming more positive, are unanticipated side effects emerging?When treatment includes neural implants, need for objective, re-peatable biometrics tied to neurophysiology becomes especiallypressing. We used Automated face analysis to assess treatmentresponse to deep Brain Stimulation (DBS) in two patients with in-tractable obsessive - compulsive disorder (OCD).

2 One was assessedintraoperatively following implantation and activation of the DBSdevice. The other was assessed three months were assessed during DBS on and off conditions. Positive andnegative valence were quantified using a CNN trained on norma-tive data of 160 non-OCD participants. Thus, a secondary goal wasdomain transfer of the classifiers. In both contexts, DBS-on resultedin marked positive Affect . In response to DBS-off, Affect flattenedin both contexts and alternated with increased negative Affect inthe outpatient setting. Mean AUC for domain transfer was findings suggest that parametric variation of DBS is stronglyrelated to affective behavior and may introduce vulnerability fornegative Affect in the event that DBS is CONCEPTS Applied computing Psychology;KEYWORDSO bsessive compulsive disorder, Deep Brain Stimulation , Facial ex-pression, Body expression, Action units, Behavioral dynamics, So-cial signal processingPermission to make digital or hard copies of all or part of this work for personal orclassroom use is granted without fee provided that copies are not made or distributedfor profit or commercial advantage and that copies bear this notice and the full citationon the first page.

3 Copyrights for components of this work owned by others than ACMmust be honored. Abstracting with credit is permitted. To copy otherwise, or republish,to post on servers or to redistribute to lists, requires prior specific permission and/or afee. Request permissions from 18, October 16 20, 2018, Boulder, CO, USA 2018 Association for Computing ISBN 978-1-4503-5692-3/18/10.. $ Reference Format:Jeffrey F. Cohn, Laszlo A. Jeni, Itir Onal Ertugrul, Donald Malone, MichaelS. Okun, David Borton, and Wayne K. Goodman. 2018. Automated AffectDetection in Deep Brain Stimulation for obsessive - compulsive Disorder:A Pilot Study . In2018 International Conference on Multimodal Interaction(ICMI 18), October 16 20, 2018, Boulder, CO, , New York, NY, USA,5 pages. INTRODUCTIONO bsessive compulsive disorder (OCD) is a persistent, oftentimesdisabling psychiatric disorder that is characterized by obsessivethoughts and compulsive behavior. obsessive thoughts are intru-sive and unwanted and can be highly disturbing.

4 Compulsions arerepetitive behaviors that an individual feels driven to perform. Theobsessive thoughts are recognized as irrational, yet they remainhighly troubling and are relieved if only temporarily by compulsiverituals, such as repetitive hand washing or repeatedcheckingto seewhether a particular event ( , shutting a door) was performed [1].When severe, individuals may be home-bound and highly impairedpersonally and in their ability to function in family or work set-tings. While cognitive behavioral therapy or medication often aresuccessful in providing relief, about25%of patients fail to respondto them. Their OCD is treatment resistant, Stimulation of the ventral striatum (VS) has proveneffective in about60%of otherwise treatment resistant ( , in-tractable) cases [16]. DBS entails implanting electrodes into theVS for continuous deep Brain Stimulation (DBS). The VS is part ofareward circuitthat is involved in appetitive behavior and emo-tion processes more generally.

5 A potential side effect of DBS fortreatment of OCD is hypomania or mania, which can have seri-ous consequences that can necessitate hospitalization. To avoidthis potential side effect and maximize treatment efficacy, optimalprogramming of DBS is of intense positive Affect frequently occursduring initial DBS and signals good prognosis. The mirth responseis related to the affective circuitry of VS. Programing DBS adjust-ments for OCD are made largely on the basis of subjective reportedemotion over a period of several months. While useful, subjectivejudgments are idiosyncratic and difficult to standardize. To max-imize treatment efficacy while minimizing potential side effects,objective, quantifiable, repeatable, and efficient biomarkers of treat-ment response to DBS are promising option is Automated measurement of facial expres-sion. Automated emotion recognition from facial expression is anactive area of research [26,29]. In clinical contexts, investigatorshave detected occurrence of depression, autism, conflict, and PTSD from visual features ( , face and body expression or movement)[7,10,18,22,25,27].

6 In the current pilot study, we explored thefeasibility of detecting changes in Affect in response to time-lockedchanges in neurophysiological challenge. We evaluated intraop-erative variation in DBS in relation to discrete facial actions andemotion valence in response to previous work in relation to psychological disorders hasfocused on detecting presence or absence of a clinical diagnosis( , depression). In contrast, our focus is synchronized variationbetween deep Brain Stimulation and affective behavior in a clinicaldisorder, OCD. We ask how closely Affect varies with parametricchanges in DBS of the ventral striatum in clinical patients. The goalis to evaluate a new measure of DBS, objective measurement offacial expression, in relation to Brain Stimulation . The long-termgoal is to evaluate the potential efficacy of objective measurementof behavior for modulating DBS to optimize treatment was available from two patients that were treated withDBS for intractable OCD.

7 One was recorded intraoperatively. Theother was recorded during interviews at about three months post-implantation of DBS electrodes. He was recorded when DBS wason and then three hours after DBS had been turned off. In bothpatients, our focus was on change in Affect as indicated by objectivefacial measurement in response to variation in contributions may be noted. One is exploration of a novel,objective, repeatable, efficient measure of emotion-relevant behav-ior in response to parametric variation in Brain activity. Emotion-relevant behavior is quantified using both discrete facial action units[14] and positive and negative valence. Action units are anatomi-cally based facial movements that individually or in combinationcan describe nearly all possible facial expressions. Using automatedface analysis, we measured facial action units (AU) relevant to emo-tion. Affective valence was measured using a priori combinationsof AU.

8 Valence is informed by circumplex models of emotion andRDoC (Research Domain Criteria) constructs of positive and nega-tive Affect . We explored an approach that maps specific AU to eachof these constructs. Two, we found strong support for the relationbetween variation in VS Stimulation and affective behavior. Thefindings suggest the hypothesis that DBS is related to increasedpositive Affect and vulnerability to negative Affect if Participants and observational proceduresPatient A was a 29-year-old woman with severe, treatment-resistantearly childhood onset OCD. Prior to implantation and treatmentwith DBS, she experienced nearly constant unwanted intrusivethoughts accompanied by the urge to perform neutralizing rituals,such as repeatedly snapping the shoulder straps of her clothing. Shewas video-recorded intraoperatively. While lightly sedated, she wasalert and able to engage in conversation with medical camera was oriented about 45 degrees above her face.

9 Herhead was secured in a stereotaxic frame for surgery. The durationof the recording was about 9 minutes. About 4 minutes of whichwas analyzable. During the remainder of the recording the camerawas zoomed out and face size proved too small for B was a 23-year-old man with severe OCD since earlychildhood. His OCD was characterized by extreme, recurrent doubtand checking. He would take hours to leave his house due to check-ing. He also suffered from perfectionism, particularly with shaving,which could take up to 2 hours to complete. Always late to every-thing, he missed large amounts of school as a child and young tried some college classes but could rarely manage to get was implanted with bilateral VC/VS Stimulation in 2002,which dramatically reduced his obsessive - compulsive behavior andimproved his quality of life. Approximately three months followingthe start of DBS treatment, he was interviewed in each of twoconditions.

10 One was with DBS on. The other was after DBS hadbeen turned off for about three hours. The video sample from theon condition was about 1 minute in duration; video from the offcondition was about 43 secs in duration. We thus were able tocompare his emotion expression both with and without VideoFor Patient A, video resolution was 640x426 with an intraoculardistance (IOD) of about 130 pixels. The sterotaxic frame occludedthe lateral eye corners and cheeks. For Patient B, video resolutionwas 320x240 with IOD of about 36 pixels, which is less than bicubic interpolation, IOD was increased to about 80 pixelsfor facial action unit Face tracking and registrationFaces in the video were tracked and normalized using real-time facealignment software that accomplishes dense 3D registration from2D videos and images without requiring person-specific training[20]. Faces were centred, scaled, and normalized to the averageinterocular (IOD) distance of the patients and in training of theclassifiers prior to their use in the Action Unit DetectionWe trained a convolutional neural network (CNN) on data from160 participants from the BP4D and BP4D+ databases [30,31].