294 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, …

1 294 IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 14, NO. 3, MARCH 2005. IMAGE Change Detection Algorithms: A Systematic Survey Richard J. Radke, Member, IEEE, Srinivas Andra, Student Member, IEEE, Omar Al-Kofahi, and Badrinath Roysam, Member, IEEE, Student Member, IEEE. Abstract Detecting regions of change in multiple images of the times. The goal is to identify the set of pixels that are signif- same scene taken at different times is of widespread interest due icantly different between the last IMAGE of the sequence and to a large number of applications in diverse disciplines, including the previous images; these pixels comprise the change mask.

2 Remote sensing, surveillance, medical diagnosis and treatment, civil infrastructure, and underwater sensing. This paper presents The change mask may result from a combination of underlying a systematic survey of the common processing steps and core factors, including appearance or disappearance of objects, mo- decision rules in modern change detection algorithms, including tion of objects relative to the background, or shape changes of significance and hypothesis testing, predictive models, the shading objects. In addition, stationary objects can undergo changes in model, and background modeling.

3 We also discuss important brightness or color. A key issue is that the change mask should preprocessing methods, approaches to enforcing the consistency of the change mask, and principles for evaluating and comparing not contain unimportant or nuisance forms of change, such the performance of change detection algorithms. It is hoped that as those induced by camera motion, sensor noise, illumination our classification of algorithms into a relatively small number of variation, nonuniform attenuation, or atmospheric absorption. categories will provide useful guidance to the algorithm designer.

4 The notions of significantly different and unimportant vary Index Terms Background modeling, change detection, change by application, which sometimes makes it difficult to directly mask, hypothesis testing, illumination invariance, mixture models, compare algorithms. predictive models, shading model, significance testing. Estimating the change mask is often a first step toward the more ambitious goal of change understanding: segmenting I. INTRODUCTION and classifying changes by semantic type, which usually re- quires tools tailored to a particular application.

5 The present D ETECTING regions of change in images of the same scene taken at different times is of widespread interest due to a large number of applications in diverse disciplines. survey emphasizes the detection problem, which is largely application independent. We do not discuss algorithms that are specialized to application-specific object classes, such as Important applications of change detection include video parts of human bodies in surveillance imagery [3] or buildings surveillance [1] [3], remote sensing [4] [6], medical diagnosis in overhead imagery [6], [21].

6 Furthermore, our interest here and treatment [7] [11], civil infrastructure [12], [13], under- is only in methods that detect changes between raw images, water sensing [14] [16], and driver assistance systems [17], as opposed to those that detect changes between hand-labeled [18]. Despite the diversity of applications, change detection region classes. In remote sensing, the latter approach is called researchers employ many common processing steps and core post-classification comparison or delta classification .1. algorithms. The goal of this paper is to present a systematic Finally, we do not address two other problems from different survey of these steps and algorithms.

7 Previous surveys of fields that are sometimes called change detection : first, the change detection were written by Singh in 1989 [19] and estimation theory problem of determining the point at which Coppin and Bauer in 1996 [20]. These articles discussed only signal samples are drawn from a new probability distribution remote-sensing methodologies. Here, we focus on more recent (see, , [24]), and second, the video processing problem of work from the broader (English-speaking) IMAGE analysis determining the frame at which an IMAGE sequence switches community that reflects the richer set of tools that have since between scenes (see, , [25], [26]).

8 Been brought to bear on the topic. We begin in Section II by formally defining the change detec- The core problem discussed in this paper is as follows. We are tion problem and illustrating different mechanisms that can pro- given a set of images of the same scene taken at several different duce apparent IMAGE change. The remainder of the survey is or- ganized by the main computational steps involved in change de- Manuscript received November 25, 2003; revised August 3, 2004. This re- tection. Section III describes common types of geometric and ra- search was supported in part by the National Science Foundation Center for diometric IMAGE preprocessing operations.

9 Section IV describes Subsurface Sensing and Imaging Systems (CenSSIS) under the Engineering Re- search Center's program of the National Science Foundation (Award Number the simplest class of algorithms for making the change decision EEC-9986821) and in part by Rensselaer Polytechnic Institute. The associate based on IMAGE differencing. The subsequent Sections V VII. editor coordinating the review of this manucript and approving it for publica- tion was Dr. Joachim M. Buhmann. 1 Such methods were shown to have generally poor performance [19], [20]. The authors are with the Department of Electrical, Computer, and We note that Deer and Eklund [22] described an improved variant where the Systems Engineering, Rensselaer Polytechnic Institute, Troy, NY, 12180 class labels were allowed to be fuzzy, and that Bruzzone and Serpico [23] de- USA (e-mail: scribed a supervised learning algorithm for estimating the prior, transition, and posterior class probabilities using labeled training data and an adaptive neural Digital Object Identifier network.)

10 1057-7149/$ 2005 IEEE. Authorized licensed use limited to: University of Illinois. Downloaded on January 12, 2009 at 13:45 from IEEE Xplore. Restrictions apply. RADKE et al.: IMAGE CHANGE DETECTION ALGORITHMS 295. Fig. 1. Apparent IMAGE changes have many underlying causes. This simple example includes changes due to different camera- and light-source positions, as well as changes due to nonrigid object motion (labeled M ), specular reflections (labeled S ), and object appearance variations (labeled A ). Deciding and detecting which changes are considered significant and which are considered unimportant are difficult problems and vary by application.