Lecture 2 Image Processing and Filtering

1 Lecture 2 Image Processing and Filtering UW CSE vision facultyWhat s on our plate today? Image formation Image sampling and quantization Image interpolation Domain transformations Affine Image transformations Range (intensity) transformations Noise reduction through spatial Filtering Filtering as cross-correlation Convolution Nonlinear (median) filteringImage Formation: Basics(from Gonzalez & Woods, 2008)f(x,y)i(x,y)r(x,y) Image Formation: BasicsImage f(x,y) is characterized by 2 components1. Illumination i(x,y)= Amount of source illumination incident on scene2. Reflectance r(x,y)= Amount of illumination reflected by objects in the scene1),(0 and ),(0where),(),(),(<< <<=yxryxiyxryxiyxfr(x,y) depends on object propertiesr = 0 means total absorption and 1 means total reflectanceImage Formation: BasicsTypical values of i(x,y): Sun on a clear day: 90,000 lm/m2 Cloudy day: 10,000 lm/m2 Inside an office: 1000 lm/m2 Typical values of r(x,y) Black velvet: , Stainless steel: , Snow: limits of f(x,y) in an office environment 10 < f(x,y) < 1000 Shifted to gray scale [0, L-1].

2 0 = black, L-1 = 255 = white1),(0 and ),(0where),(),(),(<< <<=yxryxiyxryxiyxfr=1 Sampling and Quantization Process(from Gonzalez & Woods, 2008)Example of a Quantized 2D ImageContinuous Image projected onto sensor arrayResult of sampling and quantization(from Gonzalez & Woods, 2008)Suppose we want to zoom an Image Zoomed Image Original Image Need to fillin values for new pixelsInterpolation**OriginalZoomedNeare st NeighborInterpolationNeed to fill in missing values*For each new pixel, copy nearest valueNeared Neighbor Interpolation Zoomed Image Original Image Can we do better?Other Image interpolation techniquesBilinear interpolation:Compute pixel value v(x,y) as:a, b, c, d determined from four nearest neighborsof (x,y)Bicubic interpolation:(Used in most commercial Image editing programs, , Photoshop)aijdetermined from 16 nearest neighborsof (x,y)dcxybyaxyxv+++=),(jijiijyxayxv ===3030),((from )See also of Interpolation TechniquesNearest NeighborBilinearBicubicRecall from Last TimeDomain transformation:(What is an example?)

3 TranslationRotationHow are these done?Geometric spatial transformations of imagesTwo steps:1. Spatial transformation of coordinates (x,y)2. Interpolation of intensity value at new coordinatesWe already know how to do (2), so focus on (1)Example: What does the transformation (x,y) = T((v,w)) = (v/2,w/2) do?[Shrinks original Image in half in both directions]Affine Spatial Transformations Most commonly used set of transformations General form: [x y 1] are called homogenous coordinates Can translate, rotate, scale, or shear based on values tij Multiple transformations can be concatenated by multiplying them to form new matrixT [][][] ==100 1 11323122211211ttttttwvwvyxTExample: Translation[][][] ==100 1 11323122211211ttttttwvwvyxTWhat does Tlook like for translation?

4 X = v + txy = w + tyAffine TransformationsTransformation Affine Matrix TCoordinate Equations ExampleAffine Transformations (cont.)Transformation Affine Matrix TCoordinate Equations ExampleExample of Affine TransformationImage rotated 21 degreesNearestNeighborBilinearBicubic(fr om Gonzalez & Woods, 2008)Recall from last timeRange transformation:(What is an example?)Noise filteringImage Processing for noise reductionCommon types of noise: Salt and pepper noise: contains random occurrences of black and white pixels Impulse noise: contains random occurrences of white pixels Gaussian noise: variations in intensity drawn from a Gaussian normal distributionOriginalGaussian noiseSalt and pepper noiseImpulse noiseHow do we reduce the effects of noise?

5 00000000000000000000000 909090909000000 909090909000000909090909000000900 9090900000090909090900000000000000090000 00000000000000 How do we reduce the effects of noise?00000000000000000000000 909090909000000 909090909000000 909090909000000900 90909000000 9090909090000000000000009000000000000000 00080 How do we reduce the effects of noise?00000000000000000000000 909090909000000 909090909000000 909090909000000900 90909000000 9090909090000000000000009000000000000000 0008010 Idea: Compute mean value for each pixel from neighborsMean filtering0000000000000000000000090909090 9000000 909090909000000 909090909000000900 90909000000 9090909090000000000000009000000000000000 0000 102030303020100 204060606040200 306090909060300 305080809060300 305080809060300 2030505060402010 20 30 30 30 30 20 1010101000000 Filtering as cross-correlationIf the averaging window is (2k+1)x(2k+1):In our example in previous slide, k = 1 for a 3x3 averaging windowFiltering as cross-correlationCan generalize this by allowing different weights for different neighboring pixels.

6 This is called cross-correlation, denoted by:H is called the filter, kernel, or mask. Note: During implementation, we avoid the negative filter indices by using H[u+k,v+k] instead of H[u,v]Kernel for mean filteringWhat is the kernel for a 3x3 mean filter?00000000000000000000000 909090909000000 909090909000000909090909000000900 9090900000090909090900000000000000090000 00000000000000 Kernel for mean filteringWhat is the kernel for a 3x3 mean filter?11111111100000000000000000000000 909090909000000 909090909000000909090909000000900 9090900000090909090900000000000000090000 000000000000001/9 Example of mean filteringSalt and pepper noiseInput imageFiltered Images3 x 35 x 57 x 7 Kernel sizeGaussian FilteringA Gaussian kernel gives less weight to pixels further from the center of the window00000000000000000000000 909090909000000 909090909000000 909090909000000900 90909000000 9090909090000000000000009000000000000000 000121242121 Kernel approximates Gaussian function:What happens if you increase ?

7 Mean versus Gaussian filteringInputImageMeanfilteredGaussianf ilteredFiltering an impulse000000000000000000000000100000000 0000000000000000 Impulse signalabcde fgh iKernelOutput = ? Filtering an impulse000000000000000000000000100000000 0000000000000000 Impulse signalabcde fgh iFilter KernelOutput is equal to filter kernel flipped horizontally & vertically 0000000000000000ihg0000fed0000cba0000000 000000000 What if we want to get an output that looks exactly like the filter kernel?Flipping kernels000000000000000000000000100000000 0000000000000000 Impulse signalabcde fgh iFilter KernelOutput is equal to filter kernel! 0000000000000000abc0000de f0000gh i0000000000000000ihgfedcbaFlipped KernelConvolutionA convolutionis a cross-correlation where the filter is flipped both horizontally and verticallybefore being applied to the Image :Written as:Compare with cross-correlation: If H is a Gaussian or mean kernel, how does convolution differ from cross-correlation?

8 Why convolution? Convolution is associative (cross-corr. is not):F * (G * I) = (F * G) * I Important for efficiency:To apply two filters F and G sequentially to incoming images I, pre-compute(F * G) and perform only 1 convolution (with pre-computed filter) Convolution also allows effects of Filtering to be analyzed using Fourier analysis (will touch on this later)Cross-correlation and template matchingCross-correlation is useful for template matching(locating a given pattern in an Image )abcdefghiImageabcde fgh iTemplate (pattern)Highest value yields location of pattern in imageNonlinear filters: Median filter A Median Filterreplaces the value of a pixel by the median of intensity values of neighbors Recall: m is the median of a set of values iff half the values in the set are <= m and half are >= m.

9 Median Filtering of Image I: For each location (x,y), sort intensity values in its neighborhood, determine median intensity value, and assign that value to I(x,y) Is a median filter better than a mean filter? Is median Filtering a convolution?Comparison of filters (salt-and-pepper noise)Comparison of filters (Gaussian noise)Next Time: Edge detection Things to do: Read Chap. 5: Secs. - , and online article by Cipolla & Gee on edge detection Mailing list: Did you receive the first message? Otherwise, sign up Prepare for C/C++ programming Visit Vision and Graphics Lab (Sieg 327) Your ID card should open Sieg 327 Check to make sure ASAPHave a good weekend!