Geometric Transformations: Warping, …

1 Geometric Transformations: warping , registration , MorphingMorphingYao WangPolytechnic University, Brooklyn, NY 11201 With contribution from Zhu Liu, Onur Guleryuz, and Partly based onA K Jain Fundamentals of Digital Image ProcessingA. K. Jain, Fundamentals of Digital Image ProcessingLecture Outline IntroductionImage deformation model Image deformation model Image warping Image registration Image morphingImage morphingGeometric TransformationEL512 Image Processing2 What is Geometric transformation ? So far, the image processing operations we have discussed modify thecolor valueswe have discussed modify the color valuesof pixels in a given image With Geometric transformation , we modifyWith Geometric transformation , we modify the positionsof pixels in a image, but keep their colors unchanged To create special effects To register two images taken of the same scene at different times To morph one image to anotherGeometric TransformationEL512 Image Processing3 How to define a Geometric transformation ?

2 Let (u, v) represent the image coordinate in an original image, and (x, y) in a deformed (or warped) image. We use a function pair to relate corresponding pixels in theuse a function pair to relate corresponding pixels in the two images: Forward mapping:)(,)(),(uxxorvuyyvuxx Inverse mapping:),(vuyy )(,),(),(xuuoryxvvyxuu Let f(u, v) or f(u) denote the original image and g(x, y) or g(x) the deformed image. Then they are related by: ),(yg(x) the deformed image. Then they are related by: ))(()())(()(,)),(),,((),()),(),,((),(uux xxgfufgorvuyvuxgvufyxvyxufyxgGeometric TransformationEL512 Image Processing4 Illustration of Forward and Inverse Mapping FunctionsMapping Functionsyv(u, v)Forward()()yuuvx(u,v), y(u,v)xyxuInverseu(x,y), v(x,y)x(x, y)yGeometric TransformationEL512 Image Processing5 Translation Translationis defined by the following mapping functions:vmapping functions:xxtyvtxuandtvytux v In matrix notationyytyvtvy u, wheretxutuxy(tx,ty).

3 ,, yxttvuyxtuxxGeometric TransformationEL512 Image Processing6 Scaling Scalingis defined by / Matrix notation yxyxsyvsxuandvsyusx//vysx=2,sy=1/2 Matrix notation where,1xSuSuxuxIf1d1 thiti ifi ti yxss00S If sx< 1 and sy< 1, this represents a minification or shrinking, if sx>1 and sy> 1, it represents a magnification orzoomGeometric TransformationEL512 Image Processing7magnification or Rotation by an angle of is defined by ii cossinsincoscossinsincosyxvyxuandvuyvuxv y In matrix format ,xRuRuxTvy(u,v)(x,y) = /4 cossinsincosRwhereux Ris a unitary matrix: R-1=RT cossinGeometric TransformationEL512 Image Processing8 Geometric TransformationEL512 Image Processing9By Onur GuleyuzGeometric transformation A Geometric transformationrefers to a combination oftranslationscalingandcombination of translation, scaling, and rotation, with a general form of)(bAutuRSx ,)(,)(11tcRStbRSAcxAbxAubAutuRSx with Note that interchanging the order of.

4 ,,tcRStbRSA withNote that interchanging the order of operations will lead to different TransformationEL512 Image Processing10 Affine Mapping All possible Geometric transformationsare special cases of the Affine Mapping:pp g 210210orvbubbyvauaaxbAux 021210aaavbubbybA When A is aorthonormalmatrix it corresponds to a 021 ,bbbbA When A is a orthonormalmatrix, it corresponds to a rotation matrix, and the corresponding affine mapping reduces to a Geometric TransformationEL512 Image Processing11 Affine Mapping Properties Mapping straight lines to straight lines The mapping between two arbitrary triangles The mapping between two arbitrary triangles can be captured by an affine mappingAffine mapping coefficients can be uniquely Affine mapping coefficients can be uniquely determined from displacements of 3 verticesv3y12312u2x1 How to Determine Affine Parameters from Vertex Correspondences ?

5 From Vertex Correspondences ? Go through in class v13y23u2x1 Geometric TransformationEL512 Image Processing13 Matlab Functions T = MAKETFORM('affine',U,X) builds a TFORM struct for a two-dimensional affine transformation that maps each row of Uto the corresponding row of X U and X are each 3by2 and to the corresponding row of X. U and X are each 3-by-2 and define the corners of input and output triangles. The corners may not be Example ------- Create an affine transformation that maps the triangle with vertices(0 0) (6 3) (2 5) t th t ilithti(11) (010) (0,0), (6,3), (-2,5) to the triangle with vertices (-1,-1), (0,-10), (4,4): [0 62]' u = [ 0 6 -2]'; v = [ 0 3 5]'; x = [-1 0 4]';[110 4]' y = [-1 -10 4]'; tform = maketform('affine',[u v],[x y]); Geometric TransformationEL512 Image Processing14 G = MAKETFORM('affine',T) builds a TFORM struct G for an N-dimensional affine transformation .

6 T defines a forward transformation such that TFORMFWD(U T) where U is a 1byNtransformation such that TFORMFWD(U,T), where U is a 1-by-N vector, returns a 1-by-N vector X such that X = U * T(1:N,1:N) + T(N+1,1:N).T has both forward and inverse transformations. N=2 for 2D image transformation2D image transformation 0notationMATLABInb 10100002211 TTbababaTbA 00 Geometric TransformationEL512 Image Processing15 Geometric TransformationEL512 Image Processing15 B = IMTRANSFORM(A,TFORM, INTERP) transforms the image A according to the 2-D spatial transformation defined by TFORMB; INTERP specifies the interpolation filterINTERP specifies the interpolation filter Example 1 --------- Apply a horizontal shear to an intensity image.

7 I = imread(' ');() tform = maketform('affine',[1 0 0; .5 1 0; 0 0 1]); J = imtransform(I,tform); figureimshow(I) figureimshow(J) figure, imshow(I), figure, imshow(J) Show in classGeometric TransformationEL512 Image Processing16 Horizontal Shear Exampletform=maketform('affine'[100; 510;001]);tform maketform(affine,[1 0 0; .5 1 0; 0 0 1]);In MATLAB, affine transform is defined by: [a1,b1,0;a2,b2,0;a0,b0,1]tltthiidttiWith 00 , thisin used notation With bAGeometric TransformationEL512 Image Processing17verticindicate coordinate second position, horizontal indicates coordinatefirst example, thisin NoteBilinear Mapping uvavauaax uvbvbubbyuvavauaax32103210 Mapping a square to a quadrangle Mapping a square to a quadrangle.

8 More generally mapping a quadrangle to another 8 parameters. Can be completely determined from how the four corners TransformationEL512 Image Processing18 How to determine the bilinear coefficients from the vertexcoefficients from the vertex correspondences?3v3y3424 Geometric TransformationEL512 Image Processing19u12x1 Different Types of MappingNon-chirping modelsChirping models(Original)(Affine)(Bilinear)(Proje ctive)(Relative-projective)(Pseudo-persp ective)(Biquadratic)Two features of projective mapping: Chirping: increasing perceived spatial frequency for far away objects Converging (Keystone): parallel lines converge in distance Yao Wang, 20032-D Motion Estimation, Part 120 Polynomial warping The polynomial warpingincludes all deformations that can be modeled by polynomial transformations.

9 2252432105243210vbubuvbvbubbyvauauvavaua ax Includes affine and bilinear mapping as special casesGeometric TransformationEL512 Image Processing21 Image warping by Forward Mapping Mapping image f(u, v) to g(x, y) based on a given mapping function: x(u, v), y(u, v).FdMi Forward Mapping For each point (u, v) in the original image, find the corresponding position (x, y) in the deformed image by the forward mapping function, and let g(xy)=f(uv)and let g(x,y)=f(u,v). What if the mapped position (x,y) is not an integer sample in the desired image? warping points are often non-integer samplesMany integersamples o are not assignedValuesGeometric TransformationEL512 Image Processing22 ValuesImage warping by Inverse Mapping For each point (x, y) in the image to be obtained, find its corresponding point (u, v) in the original image using the inverse mapping function, and let g(x, y) = f(u, v).

10 What if the mapped point (u,v) is not an integer sample?Interpolate from nearby integer samples! Interpolate from nearby integer samples!P P1P2P3P4 Geometric TransformationEL512 Image Processing2324P will be interpolatedfrom P1, P2, P3, and P4 Interpolation Method Nearest neighbor: Round (uv) to the nearest integ