Lecture 12: Camera Projection - Pennsylvania State University

1 CSE486, Penn StateRobert CollinsLecture 12: Camera ProjectionReading: T&V Section , Penn StateRobert CollinsImaging GeometryVUWO bject of Interestin World CoordinateSystem (U,V,W)CSE486, Penn StateRobert CollinsImaging GeometryZfCamera Coordinate System (X,Y,Z). Z is optic axis Image plane located f unitsout along optic axis f is called focal lengthXYCSE486, Penn StateRobert CollinsImaging GeometryVUWZF orward Projection onto image (X,Y,Z) projected to 2D (x,y)yxXYCSE486, Penn StateRobert CollinsImaging GeometryVUWZyOur image gets digitizedinto pixel coordinates (u,v)xXYuvCSE486, Penn StateRobert CollinsImaging GeometryVUWZyWorld CoordinatesCameraCoordinatesImage (film)CoordinatesPixelCoordinatesuvxXYCS E486, Penn StateRobert CollinsForward ProjectionUVWXYZ xyuvWorldCoordsCameraCoordsFilmCoordsPix elCoordsWe want a mathematical model to describehow 3D World points get projected into 2 DPixel goal: describe this sequence of transformations by a big matrix equation!

2 CSE486, Penn StateRobert CollinsBackward ProjectionUVWXYZ xyuvWorldCoordsCameraCoordsFilmCoordsPix elCoordsNote, much of vision concerns trying toderive backward Projection equations torecover 3D scene structure from images (via stereo or motion)But first, we have to understand forward , Penn StateRobert CollinsForward ProjectionUVWXYZ xyuvWorldCoordsCameraCoordsFilmCoordsPix elCoords3D-to-2D Projection perspective projectionWe will start here in the middle, since we ve alreadytalked about this when discussing , Penn StateRobert CollinsBasic Perspective ProjectionXXYYZZffOOp = p = (x,y,f)xxyyZYfyZXfx , PSUZP =P =(X,Y,Z)yScene PointImage PointPerspective Projection EqnsYXxCSE486, Penn StateRobert CollinsBasic Perspective ProjectionXXYYZZffOOp = p = (x,y,f)xxyyZYfyZXfx , PSUXZP =P =(X,Y,Z)xyScene PointImage PointPerspective Projection EqnsYXxZfderived via similar triangles ruleCSE486, Penn StateRobert CollinsBasic Perspective ProjectionXXYYZZffOOp = p = (x,y,f)xxyyZYfyZXfx , PSUXZP =P =(X,Y,Z)xyScene PointImage PointPerspective Projection EqnsYXxZfZYyfderived via similar triangles ruleCSE486, Penn StateRobert CollinsBasic Perspective ProjectionXXYYZZffOOp = p = (x,y,f)xxyyZYfyZXfx , PSUXZP =P =(X,Y,Z)xyScene PointImage PointPerspective Projection EqnsYSo how do we represent this as a matrix equation?

3 We need to introduce homogeneous , Penn StateRobert CollinsHomogeneous CoordinatesRepresent a 2D point (x,y) by a 3D point (x ,y ,z ) byadding a fictitious third convention, we specify that given (x ,y ,z ) we canrecover the 2D point (x,y) as'''' zyyzxx Note: (x,y) = (x,y,1) = (2x, 2y, 2) = (k x, ky, k) for any nonzero k (can be negative as well as positive)CSE486, Penn StateRobert CollinsPerspective Matrix Equation(in Camera Coordinates) 10001000000''' ZYXffzyxZYfyZXfx CSE486, Penn StateRobert CollinsForward ProjectionUVWXYZ xyuvWorldCoordsCameraCoordsFilmCoordsPix elCoordsRigid Transformation (rotation+translation)between world and Camera coordinate systemsCSE486, Penn StateRobert CollinsWorld to Camera TransformationXXYYZZPCUUVVWWPWA void confusion: Pw and Pc are not two different points. They are the same physical point, described in two different coordinate , Penn StateRobert CollinsWorld to Camera TransformationXXYYZZPCUUVVWWPWT ranslate by - C (align origins)CRotate toalign axesRPC= R ( PW- C )CSE486, Penn StateRobert CollinsMatrix Form, Homogeneous CoordsPC= R ( PW- C ) 1000100100zyxccc0011 WVU 1000r13r12r11 r23r22r21 r33r32r311 ZYXCSE486, Penn StateRobert CollinsExample.

4 Simple Stereo SystemXYZlocated atlocated at(0,0,0)(0,0,0)leftcamerazzxxyy(X,Y,Z)T Txxrightcameralocated atlocated at(T(Txx,0,0),0,0)zzxxyy( , )( , )Left Camera located at world origin (0,0,0)and Camera axes aligned with world coord , Penn StateRobert Collins 1000100100zyxccc001 1000r13r12r11 r23r22r21 r33r32r311 ZYX1 WVUS imple Stereo, Left Camera1 0 00 1 00 0 1camera axes alignedwith world axes000located at world position (0,0,0)=CSE486, Penn StateRobert CollinsSimple Stereo Projection EquationsLeft cameraCSE486, Penn StateRobert CollinsExample: Simple Stereo SystemXYZlocated atlocated at(0,0,0)(0,0,0)leftcamerazzxxyy(X,Y,Z)T Txxrightcameralocated atlocated at(T(Txx,0,0),0,0)zzxxyy( , )( , )Right Camera located at world location (Tx,0,0)and Camera axes aligned with world coord , Penn StateRobert Collins 1000100100zyxccc001 1000r13r12r11 r23r22r21 r33r32r311 ZYX1 WVUS imple Stereo, Right Camera1 0 00 1 00 0 1camera axes alignedwith world axes-Tx00located at world position (Tx,0,0)=CSE486, Penn StateRobert CollinsRight cameraSimple Stereo Projection EquationsLeft cameraCSE486, Penn StateRobert CollinsBob s sure-fire way(s) tofigure out the rotation 1000100100zyxccc0011 WVU 1000r13r12r11 r23r22r21 r33r32r311 ZYXPC= R PWforget about thiswhile thinkingabout rotationsThis equation says how vectors in the world coordinate system (including the coordinate axes)

5 Get transformed into the Camera coordinate , Penn StateRobert CollinsFiguring out Rotations1 WVU 1000r13r12r11 r23r22r21 r33r32r311 ZYXPC= R PWwhat if world x axis (1,0,0) corresponds to Camera axis (a,b,c)?1 WVU 1000r13r12r11 r23r22r21 r33r32r311 ZYX100abc1 WVU 1000r13r12r11 r23r22r21 r33r32r311 ZYX100abcabcwe can immediately write down the first column of R!CSE486, Penn StateRobert CollinsFiguring out Rotations1 WVU 1000r13r12r11 r23r22r21 r33r32r311 ZYXand likewise with world Y axis and world Z is world coordssame axis in Camera coordsworld X axis (1,0,0)in Camera coordsworld Y axis (0,1,0)in Camera coordsworld Z axis (0,0,1)in Camera coordsCSE486, Penn StateRobert CollinsFiguring out Rotations1 WVU 1000r31r21r11 r32r22r12 r33r23r131 ZYXA lternative approach: sometimes it is easier to specifywhat Camera X,Y,or Z axis is in world coordinates. Thendo rearrange the equation as R PWR-1PC= PWRTPC= PWCSE486, Penn StateRobert CollinsFiguring out Rotationswhat if Camera X axis (1,0,0) corresponds to world axis (a,b,c)?

6 1 WVU 1000r31r21r11 r32r22r12 r33r23r131 ZYXRTPC= PW1 WVU 1000r31r21r11 r32r22r12 r33r23r131 ZYX100abcwe can immediately write down the first column of RT,(which is the first row of R).1 WVU 1000r31r21r11 r32r22r12 r33r23r131 ZYX100abcabcCSE486, Penn StateRobert CollinsFiguring out Rotations1 WVU 1000r13r12r11 r23r22r21 r33r32r311 ZYXand likewise with Camera Y axis and Camera Z is world coordssame axis in Camera coordscamera X axis (1,0,0)in world coordscamera Y axis (0,1,0)in world coordscamera Z axis (0,0,1)in world coordsCSE486, Penn StateRobert CollinsExamplexyz0 0 10 -1 01 0 0 Rtrain0 -1 00 0 1-1 0 0 RflyCSE486, Penn StateRobert CollinsNote: External Parametersalso often written as R,T 1000100100zyxccc0011 WVU 1000r13r12r11 r23r22r21 r33r32r311 ZYXtx1000r13r12r11r23r22r21r33r32r31tytz R ( PW- C )= R PW- R C= R PW+ TCSE486, Penn StateRobert CollinsSummaryUVWXYZ xyuvWorldCoordsCameraCoordsFilmCoordsPix elCoordsWe now know how to transform 3D world coordinate points into Camera coords, and then do perspective project to get 2D pointsin the film time: pixel coordinates