Poisson Image Editing - Department of Computer Science

1 Poisson Image EditingPatrick P erez Michel Gangnet Andrew Blake Microsoft Research UKAbstractUsing generic interpolation machinery based on solving Poissonequations, a variety of novel tools are introduced for seamless edit-ing of Image regions. The rst set of tools permits the seamlessimportation of both opaque and transparent source Image regionsinto a destination region. The second set is based on similar math-ematical ideas and allows the user to modify the appearance of theimage seamlessly, within a selected region. These changes can bearranged to affect the texture, the illumination, and the color of ob-jects lying in the region, or to make tileable a rectangular [ Computer Graphics]: Picture/ImageGeneration Display algorithms; [ Computer Graphics]:Methodology and Techniques Interaction techniques; [Im-age Processing and Computer Vision]: Enhancement Filtering;Keywords:Interactive Image Editing , Image gradient, guided in-terpolation, Poisson equation, seamless cloning, selection editing1 IntroductionImage Editing tasks concern either global changes (color/intensitycorrections, lters, deformations) or local changes con ned to a se-lection.

2 Here we are interested in achieving local changes, onesthat are restricted to a region manually selected, in a seamless andeffortless manner. The extent of the changes ranges from slight dis-tortions to complete replacement by novel content. classic tools toachieve that include Image lters con ned to a selection, for slightchanges, and interactive cut-and-paste with cloning tools for com-plete replacements. With these classic tools, changes in the selectedregions result in visible seams, which can be only partly hidden,subsequently, by feathering along the border of the selected propose here a generic machinery from which different toolsfor seamless Editing and cloning of a selection region can be de-rived. The mathematical tool at the heart of the approach is thePoisson partial differential equation with Dirichlet boundary con-ditions which speci es the Laplacian of an unknown function overthe domain of interest, along with the unknown function values overthe boundary of the domain.

3 The motivation is , it is well known to psychologists [Land and McCann 1971]that slow gradients of intensity, which are suppressed by the Lapla-cian operator, can be superimposed on an Image with barely notice- e-mail: effect. Conversely, the second-order variations extracted bythe Laplacian operator are the most signi cant , a scalar function on a bounded domain is uniquely de- ned by its values on the boundary and its Laplacian in the Poisson equation therefore has a unique solution and this leadsto a sound , given methods for crafting the Laplacian of an unknownfunction over some domain, and its boundary conditions, the Pois-son equation can be solved numerically to achieve seamless llingof that domain. This can be replicated independently in each ofthe channels of a color Image .

4 Solving the Poisson equation alsohas an alternative interpretation as a minimization problem: it com-putes the function whose gradient is the closest, in theL2-norm, tosome prescribed vector eld theguidancevector eld undergiven boundary conditions. In that way, the reconstructed functioninterpolates the boundary conditions inwards, while following thespatial variations of the guidance eld as closely as possible. Sec-tion 2 details this guided will examine a number of possible choices for the guidancevector eld. We show in particular that this interpolation machineryleverages classic cloning tools, both in terms of ease of use and ca-pabilities. The resulting cloning allows the user to remove and addobjects seamlessly. By mixing suitably the gradient of the sourceimage with that of the destination Image , it also becomes possibleto add transparent objects convincingly.

5 Furthermore, objects withcomplex outlines including holes can be added automatically with-out the need for painstaking cutting out. These different cloningfacilities are presented in Section shown in Section 4, the same machinery can also be usedto modify the appearance of an Image within a restricted domain,while avoiding visible discontinuities on the domain boundary. Inparticular, the color, the texture, or the illumination of an objectcan easily be modi ed without any need for precise delineation ofobject boundaries. Also, a rectangular Image region can be madeseamlessly workThe Poisson equation has been used extensively incomputer vision. It arises naturally as a necessary condition in thesolution of certain variational problems. In the speci c context ofimage Editing applications three previous pieces of work are relatedto the use of the Poisson equation proposed [Fattal et al.]

6 2002], the gradient eld of a High DynamicRange (HDR) Image is rescaled non-linearly, producing a vector eld that is no longer a gradient eld. A new Image is then ob-tained by solving a Poisson equation with the divergence of thisvector eld as right-hand-side and under Neumann boundary con-ditions specifying that the value of the gradient of the new imagein the direction normal to the boundary is zero. In contrast, themethod we are proposing here can be applied to arbitrary patchesselected from an Image , not just to the entire Image . In order to dothis, Neumann boundary conditions on a rectangular outline mustbe replaced by Dirichlet conditions on an arbitrary outline. A fur-ther generalization is to extend the range of nonlinear operationsapplied to gradients, to include maximum operations and suppres-sion of small gradients, both of which have useful Editing [Elder and Goldberg 2001], a system is introduced to edit animage via a sparse set of its edge elements (edgels).

7 To suppress anPermission to make digital/hard copy of part of all of this work for personal orclassroom use is granted without fee provided that the copies are not made ordistributed for profit or commercial advantage, the copyright notice, the title of thepublication, and its date appear, and notice is given that copying is by permissionof ACM, Inc. To copy otherwise, to republish, to post on servers, or to redistributeto lists, requires prior specific permission and/or a fee. 2003 ACM 0730-0301/03/0700-0313 $ , associated edgels are removed; to add an object, associatededgels as well as color values on both sides of each of these edgelsare incorporated. The new Image is then obtained by interpolat-ing smoothly the colors associated to the new set of edgels. Thisamounts to solving a Laplace equation (a Poisson equation with anull right hand side) with Dirichlet boundary conditions given bycolors around edgels.

8 Editing edgels and associated colors is notalways simple. In addition, Image details are lost when convertingto and from the contour domain, which might be undesirable. Thesparse edgel-based representation is indeed incomplete, as opposedto the related representation based on wavelet extrema [Mallat andZhong 1992], which are complete but less adapted to manual [Lewis 2001], spots are removed from fur images by separat-ing out the brightness component from details in a selected regionand replacing the brightness by harmonic interpolation (solving aLaplace equation) of the brightness at the selection terms of Image Editing functionalities, two existing techniquesachieve seamless cloning as the basic instance of our system rst one is Adobec Photoshopc 7's Healing Brush [Adobec 2002]. To the best of our knowledge, the technique used by thistool has not been published.

9 Therefore, we don't know whether ornot it uses a Poisson second technique is the multiresolution Image blending pro-posed in [Burt and Adelson 1983]. The idea is to use a multires-olution representation, namely a Laplacian pyramid, of the imagesof interest. The content of the source Image region is mixed, withineach resolution band independently, with its new surrounding inthe destination Image . The nal composite Image is then recov-ered by adding up the different levels of the new composite Lapla-cian pyramid thus obtained. The technique results in multiresolu-tion mixing where nest details are averaged very locally aroundthe boundary of the selection, while lower frequencies are mixedover much larger distances around these boundaries. This fast tech-nique achieves an approximate insertion of the source Laplacianin the destination region (on the rst level of the Laplacian pyra-mid) whereas we perform this Laplacian insertion exactly via thesolution of a Poisson equation.

10 More importantly, multiresolutionblending incorporates data from distant source and destination pix-els, via the upper levels of the pyramid, within the nal compositeimage. This long range mixing, which might be undesirable, doesnot occur in our technique. In addition, our system offers extendedfunctionality besides opaque seamless cloning, see Sections 3 , whereas we propose a guided interpolation framework,with the guidance being speci ed by the user, , in the form ofa source Image in the case of seamless cloning, various interpo-lation methods have been proposed to ll in Image regions auto-matically using only the knowledge of the boundary rst class of such approaches is composed of inpainting tech-niques [Ballester et al. 2001; Bertalmio et al. 2000] where PDE-based interpolation methods are devised such as to continue theisophotes hitting the boundary of the selected region.