11.1 Camera matrix - Carnegie Mellon University

1 Camera Matrix16-385 Computer Vision (Kris Kitani) Carnegie Mellon University2D to 2D Transform (last session)3D object2D to 2D Transform (last session)3D to 2D Transform (today)A Camera is a mapping between the 3D world and a 2D imagex=PXcamera matrix3D world point2D image pointWhat do you think the dimensions are?A Camera is a mapping between the 3D world and a 2D imagex=PX24 XYZ35=24p1p2p3p4p5p6p7p8p9p10p11p1235266 4 XYZ13775homogeneous world point 4 x 1homogeneous image 3 x 1 Camera matrix 3 x 4camera centerXimage planexprincipal axisThe pinhole cameraxyzz=fWhat is the equation for image coordinate x (in terms of X)?principal pointfyzXYZimage plane?What is the equation for image coordinate x (in terms of X)?fyzXfYZYZ image plane[XYZ]>7![fX/ZfY/Z]> Camera centerXimage planexprincipal axisPinhole Camera geometryxyzz=fx=PXWhat is the Camera matrix P for a pinhole Camera model?principal point24 XYZ35=24p1p2p3p4p5p6p7p8p9p10p11p1235266 4 XYZ13775[XYZ]>7!

2 [fX/ZfY/Z]>Relationship from similar Camera modelWhat does the pinhole Camera model look like?P=24f0000f00001035 What does the pinhole Camera model look like?24 XYZ35=24p1p2p3p4p5p6p7p8p9p10p11p1235266 4 XYZ13775[XYZ]>7![fX/ZfY/Z]>Relationship from similar Camera modelP=24f0000f00001035 Camera origin and image origin might be differentcamera coordinate systemimage coordinate systemCCD arraypP=24f0px00fpy000 1 035 Accounts for different originscamera coordinate systemimage coordinate systemCCD arraypXworld pointOcameraOimageIn general, the Camera and image sensor have different coordinate systemsximage pointXworld pointOcameraOimageOworldIn general, there are three different coordinate pointso you need the know the transformations between themP=24f0px0fpy00 1352410000100001035P=K[I|0]Can be decomposed into two matricesK=24f0px0fpy00 135calibration matrix (3 x 3)(3 x 4)P=24f0px0fpy00 1352410000100001035 Assumes that the Camera and world share the same coordinate systemWhat if they are different?

3 How do we align them?xcyczczwxwywWorld coordinate systemCamera coordinate systemP=24f0px0fpy00 1352410000100001035 Assumes that the Camera and world share the same coordinate systemxcyczczwxwywRtWorld coordinate systemCamera coordinate system3R rotation and translation to align axisWhat if they are different? How do we align them?xcyczczwxwywWorld coordinate systemCamera coordinate systemXwxcyczczwxwywWorld coordinate systemCamera coordinate systemXwCCoordinate of the Camera center in the world coordinate framexcyczczwxwywWorld coordinate systemCamera coordinate systemXwCCoordinate of the Camera center in the world coordinate frame(Xw C)(Xw C)TranslateXcxcyczczwxwywWorld coordinate systemCamera coordinate systemXwWhy aren t the points aligned?C(Xw C)TranslateXcR(Xw C)What happens to points after alignment?xcyczczwxwywWorld coordinate systemCamera coordinate systemXwCRXcRotateTranslateRotateXc=R(Xw C)Optionally in homogeneous coordinates:In inhomogeneous coordinates:2664 XcYcZc13775= R RC01 2664 XwYwZw13775 General mapping of a pinhole cameraP=KR[I| C](4 4)QuizWhat is the meaning of each matrix of the Camera matrix decomposition?

4 P=KR[I| C]3x3 intrinsicsQuizWhat is the meaning of each matrix of the Camera matrix decomposition?P=KR[I| C]3x3 intrinsics3x3 3D rotationQuizWhat is the meaning of each matrix of the Camera matrix decomposition?P=KR[I| C]3x3 intrinsics3x3 3D rotation3x3 identityQuizWhat is the meaning of each matrix of the Camera matrix decomposition?P=KR[I| C]3x3 intrinsics3x3 3D rotation3x3 identity3x1 3D translationGeneral mapping of a pinhole cameraP=KR[I| C]Another way to write the mappingt= RCwhere(translate first then rotate)(rotate first then translate)P=K[R|t]QuizThe Camera matrix relates what two quantities?QuizThe Camera matrix relates what two quantities?x=PXQuizThe Camera matrix relates what two quantities?x=PX3D points to 2D image pointsQuizThe Camera matrix relates what two quantities?x=PX3D points to 2D image pointsThe Camera matrix can be decomposed into?QuizThe Camera matrix relates what two quantities?

5 X=PX3D points to 2D image pointsThe Camera matrix can be decomposed into?P=K[R|t]QuizThe Camera matrix relates what two quantities?x=PX3D points to 2D image pointsThe Camera matrix can be decomposed into?P=K[R|t]intrinsic and extrinsic parametersP=24f0px0fpy00 13524r1r2r3t1r4r5r6t2r7r8r9t335 Generalized pinhole Camera modelP=K[R|t]R=24r1r2r3r4r5r6r7r8r935t=2 4t1t2t335extrinsic parametersintrinsic parameters3D rotation3D translationWhy do we need P?to properly relate world points to image points (by taking into account different coordinate systems)