PERSONAL 3D AUDIO SYSTEM WITH LOUDSPEAKERS - …

1 PERSONAL 3D AUDIO SYSTEM with LOUDSPEAKERSM yung-Suk Song#1, Cha Zhang 2, Dinei Florencio 3, and Hong-Goo Kang#4#Department of Electrical and Electronic, Yonsei University Microsoft 3D AUDIO systems often have a limited sweet spot forthe user to perceive 3D effects successfully. In this paper, wepresent a PERSONAL 3d audio system with loudspeakers that hasunlimited sweet spots. The idea is to have a camera track theuser s head movement, and recompute the crosstalk cancellerfilters accordingly. As far as the authors are aware of, our sys-tem is the first non-intrusive 3D AUDIO SYSTEM that adapts to boththe head position and orientation with six degrees of effectiveness of the proposed SYSTEM is demonstrated withsubjective listening tests comparing our SYSTEM against tradi-tional non-adaptive binaural, immersive, 3D AUDIO , head tracking1.

2 INTRODUCTIONA three-dimensional AUDIO SYSTEM renders sound images arounda listener by using either headphones or LOUDSPEAKERS [1]. Inthe case of a headphone-based 3D AUDIO SYSTEM , the 3D cuesto localize a virtual source can be perfectly reproduced at thelistener s ear drums, because the headphone isolates the listenerfrom external sounds and room reverberations. In contrast, withloudspeakers, the sound signal from both speakers will be heardby both ears, which creates challenges for generating 3D simple yet effective technique for loudspeaker -based 3 Daudio isamplitude panning[2]. Amplitude panning relies on thefact that human can perceive sound directions effectively basedon the level difference between the ear drums.

3 It renders thevirtual sound source at different locations by adaptively control-ling the output amplitude of the LOUDSPEAKERS . Unfortunately,amplitude panning cannot reproduce virtual sources outside theregion of LOUDSPEAKERS , which limits its applications in desktopscenarios where usually only two LOUDSPEAKERS are alternative solution is to generate the virtual soundsources based on synthetic head related transfer functions(HRTF) [3] through crosstalk cancellation. Crosstalk cancel-lation uses the knowledge of HRTF and attempts to cancel thecrosstalk between the left loudspeaker and the right ear and be-tween the right loudspeaker and the left ear.

4 Since HRTF faith-fully records the transfer function between sound sources andhuman ears, the virtual sound source can be placed beyond theFig. 1. Our PERSONAL 3D AUDIO SYSTEM with one webcam on thetop of the monitor, and two boundaries. On the other hand, HRTF varies dueto changes in head positions and orientations, thus such HRTF-based 3D AUDIO systems work only when the user is in a smallzone called sweet spot .In order to overcome the small sweet spot problem, re-searchers have proposed to use a head tracking module to fa-cilitate 3D AUDIO generation [4, 5, 6, 7]. The listener s headmovement is tracked to adaptively control the crosstalk can-celler in order to steer the sweet spot towards the user s headposition/orientation.

5 For instance, in [8, 9], the listener s headmovement was tracked using electromagnetic trackers, althoughsuch devices are expensive and discomfortable to wear. A non-intrusive and more attractive method is to track the head move-ment with webcams and face tracking techniques [5, 10, 11].Nevertheless, due to the limited computational resources and in-capable face tracking techniques at that time, these early workscannot fully evaluate the effectiveness of tracking based 3D au-dio generation. For instance, none of the above work consideredthe listener s movement beyond their 2D motion parallel to thewebcam s imaging plane, and none of them provided any eval-uation results on how well their systems this paper, we combine a 3D model based face tracker withdynamic binaural synthesis and dynamic crosstalk cancellationto build a true PERSONAL 3D AUDIO SYSTEM .

6 The basic hardware978-1-4244-7493-6/10/$ 2010 IEEEICME 2010 Fig. 2. Schematic of binaural AUDIO SYSTEM with is shown in Figure 1. The webcam-based 3D face trackerprovides accurate head position and orientation information tothe binaural AUDIO SYSTEM , which uses the information to adap-tively synthesize the target AUDIO to be played by the loudspeak-ers. The SYSTEM runs in real-time on a dual-core 3 GHz machine,which serves the listener with realistic 3D auditory addition, we conducted subjective listening tests to eval-uate the effectiveness of head tracking for 3D AUDIO were asked to identify the virtual sound source loca-tions at different head positions.

7 The results were comparedwith the ground truth information to measure the impact of headtracking on human localization accuracy. Results of the subjec-tive tests showed clear advantage of the proposed SYSTEM whencompared with traditional 3D AUDIO systems without head track-ing based rest of the paper is organized as follows. Section 2 in-troduces conventional binaural AUDIO systems. The proposedpersonal 3D AUDIO SYSTEM with head tracking is described inSection 3. Experimental results and conclusions are presentedin Section 4 and Section 5, CONVENTIONAL BINAURAL AUDIO SYSTEMThe block diagram of a typical binaural AUDIO playback systemwith two LOUDSPEAKERS is depicted in Figure 2.

8 ComponentCrepresents the physical transmission path or the acoustic chan-nel between the LOUDSPEAKERS and the listener s ears, which isusually assumed as known. The binaural AUDIO SYSTEM consistsof two major blocks: binaural synthesizerBand crosstalk can-cellerH. The goal of the binaural synthesizer is to producesounds that should be heard by the listener s ear drums. In otherwords, we hope the signals at the listener s earseLandeRshallbe equal to the binaural synthesizer outputxLandxR. Thecrosstalk canceller, subsequently, aims to equalize the effect ofthe transmission pathC[12][13]. Binaural synthesisThe binaural synthesizerBsynthesizes one or multiple virtualsound images at different locations around the listener using 3 Daudio cues.

9 Among many binaural cues for the human auditorysystem to localize sounds in 3D such as the interaural time dif-ference (ITD) and the interaural intensity difference (IID), weexplore the use of HRTF, which is the Fourier transform of thehead-related impulse response (HRIR). Since HRTF capturesFig. 3. acoustic path between two LOUDSPEAKERS and listener searsmost of the physical cues that human relies on for source local-ization. Once the HRTFs of the ears are known, it is possibleto synthesize accurate binaural signals from a monaural source[4]. For instance, one can filter the monaural input signal withthe impulse response of the HRTF for a given angle of incidenceas:x=[xLxR]=[BLBR]x=Bx,(1)where xis the monaural input signal,BLandBRare the HRTF sbetween the listener s ears and the desired virtual source.

10 Theoutput of binaural synthesisxLandxRare the signals thatshould be reproduced at the listener s ear Crosstalk CancellationThe acoustic paths between the LOUDSPEAKERS and the listener sears (Figure 3) are described by an acoustic transfer matrixC:C=[CLLCRLCLRCRR],(2)whereCLLis the transfer function from the left speaker to theleft ear, andCRRis the transfer function from the right speakerto the right ear. For headphone applications, the acoustic chan-nels are completely separated, because the sound signal fromthe left speaker goes only to the left ear, and the right signalgoes only to the right ear.