Example: tourism industry

Self-Adaptive Matrix Completion for Heart Rate Estimation ...

Self-Adaptive Matrix Completion for Heart Rate Estimation from Face Videos under Realistic Conditions Sergey Tulyakov1 , Xavier Alameda-Pineda1 , Elisa Ricci2,3 , Lijun Yin4 , Jeffrey F. Cohn5,6 , Nicu Sebe1. 1. University of Trento, Via Sommarive 9, 38123 Trento, Italy 2. Fondazione Bruno Kessler, Via Sommarive 18, 38123 Trento, Italy 3. University of Perugia, Via Duranti 93, 06123, Perugia, Italy 4. State University of New York at Binghamton, Binghamton, NY 13902, USA. 5. Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA.

Self-Adaptive Matrix Completion for Heart Rate Estimation from Face Videos under Realistic Conditions Sergey Tulyakov1, Xavier Alameda-Pineda1, Elisa Ricci2,3, Lijun Yin4, Jeffrey F. Cohn5,6, Nicu Sebe1 1University of Trento, Via Sommarive 9, 38123 Trento, Italy 2Fondazione Bruno Kessler, Via Sommarive 18, 38123 Trento, Italy 3University of Perugia, Via Duranti 93, 06123, Perugia, Italy

Tags:

  Earth, Rates, Adaptive, Matrix, Completion, Estimation, Adaptive matrix completion for heart rate estimation

Information

Domain:

Source:

Link to this page:

Please notify us if you found a problem with this document:

Other abuse

Transcription of Self-Adaptive Matrix Completion for Heart Rate Estimation ...

1 Self-Adaptive Matrix Completion for Heart Rate Estimation from Face Videos under Realistic Conditions Sergey Tulyakov1 , Xavier Alameda-Pineda1 , Elisa Ricci2,3 , Lijun Yin4 , Jeffrey F. Cohn5,6 , Nicu Sebe1. 1. University of Trento, Via Sommarive 9, 38123 Trento, Italy 2. Fondazione Bruno Kessler, Via Sommarive 18, 38123 Trento, Italy 3. University of Perugia, Via Duranti 93, 06123, Perugia, Italy 4. State University of New York at Binghamton, Binghamton, NY 13902, USA. 5. Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213, USA.

2 6. Department of Psychology, University of Pittsburgh, Pittsburgh, PA 15260, USA. Abstract Recent studies in computer vision have shown that, while practically invisible to a human observer, skin color changes due to blood flow can be captured on face videos and, surprisingly, be used to estimate the Heart rate (HR). While considerable progress has been made in the last few years, still many issues remain open. In particular, state- of-the-art approaches are not robust enough to operate in natural conditions ( in case of spontaneous movements, facial expressions, or illumination changes).

3 Opposite to Time previous approaches that estimate the HR by processing all Figure 1. Motivation: Given a video sequence, automatic HR es- the skin pixels inside a fixed region of interest, we intro- timation from facial features is challenging due to target motion duce a strategy to dynamically select face regions useful for and facial expressions. Facial features extracted over time in dif- robust HR Estimation . Our approach, inspired by recent ad- ferent parts of the face (purple rectangles) show different temporal vances on Matrix Completion theory, allows us to predict dynamics and are subject to noise, as they are heavily affected by the HR while simultaneously discover the best regions of movements and illumination changes.

4 In this paper, we propose a the face to be used for Estimation . Thorough experimental novel approach to simultaneously estimate the HR signal and se- evaluation conducted on public benchmarks suggests that lect the reliable face regions at each time for robust HR prediction. the proposed approach significantly outperforms state-of- the-art HR Estimation methods in naturalistic conditions. nificant improvements in face tracking and alignment meth- ods [3, 21, 13, 14, 29], facial-based remote Heart rate esti- mation has recently become very popular [17, 30, 10, 25].

5 1. Introduction Classical approaches successfully addressed this prob- After being shown in [23, 18] that changes invisible to lem under laboratory-controlled conditions, imposing the naked eye can be used to estimate the Heart rate from constraints on the subject's movements and requiring the a video of human skin, this topic has attracted a lot of at- absence of facial expressions and mimics [18, 27, 4]. There- tention in the computer vision community. These subtle fore, such methods may not be suitable for real world appli- changes encompass both color [27] and motion [4] and they cations, such as monitoring drivers inside a vehicle or peo- are induced by the internal functioning of the Heart .

6 Since ple exercising. Long-time analysis constitutes a further lim- faces appear frequently in videos and due to recent and sig- itation of existing works [17, 18, 19]. Indeed, instead of es- 12396. timating the instantaneous Heart rate, they provide the aver- show that our method outperforms the state-of-the-art ap- age HR measurement over a long video sequence. The main proaches for HR prediction. To further demonstrate the disadvantage of using a long analysis window is the inabil- ability of our method to operate in challenging scenar- ity to capture interesting short-time phenomena, such as a ios, we report a series of tests on the MMSE-HR dataset, sudden HR increase/decrease due to specific emotions [22].

7 Where subjects show significant movements and facial ex- In practice, another problem faced by researchers de- pressions. veloping automatic HR measurement approaches, is the Thus, the contribution of this paper is three-fold: lack of publicly available datasets recorded under realis- tic conditions. A notable exception is the MAHNOB-HCI We present a novel approach to address the problem of dataset [20], a multimodal dataset for research on emotion HR Estimation from face videos in realistic conditions.

8 Recognition and implicit tagging, which also contains HR To cope with large facial variations due to spontaneous annotations. Importantly, an extensive evaluation of ex- facial expressions and movements, we propose a prin- isting HR measurement methods on MAHNOB-HCI have cipled framework to automatically discard the face re- been performed by Li et al. [17]. However, the MAHNOB- gions corresponding to noisy features and only use the HCI dataset suffers from some limitations, since the record- reliable ones for HR prediction.

9 The region selection ing conditions are quite controlled: most of the video se- is addressed within a novel Matrix Completion -based quences do not contain spontaneous facial expressions, illu- optimization framework, called Self-Adaptive Matrix mination changes or large target movements [17]. Completion , for which an efficient solver is proposed. In this work, we tackle the aforementioned problems Our approach is demonstrated to be more accurate than by introducing a novel approach for HR Estimation from previous methods for average HR Estimation on pub- face videos and providing an extensive evaluation on two licly available benchmarks.

10 In addition, we report datasets: the MAHNOB-HCI, previously used for HR short-term analysis results to show the ability of our recognition research [17], and a spontaneous dataset with method to detect instantaneous Heart rate. Heart rate data and RGB videos (named MMSE-HR), which is a subset of the larger multimodal spontaneous emotion We perform extensive evaluation on the commonly corpus (MMSE) [31] specifically targeted to challenge HR used MAHNOB-HCI dataset and a spontaneous Estimation methods.


Related search queries