視覚の幾何学3 カメラキャリブレーション

1 1 Rigid Body Motion Two views Xx 11 txRx 1122 Xx 22 babbaba 000;121323122131132332321321aaaaaabababa babababbbaaa 00 b)(abb)(aaba matrix form of cross product Geometric transformation ] with ]0| with t|[RE'pE'p'[IEEpptRpp' ] with ]0| with t|[RP'xP'x'[IPPxxtRxx' R : 3*3 rotation matrix t : 3*1 translation vector p p (Essential matrix ) p p ( ) TTyxyx)1,','(')1,,( with 0)](['ppRptpFrom geometry to algebra TRpp' TTRpTpT 'RpT RpTppTp '''0 Normal to the planeSlide credit: Kristen GraumanP2 R : 3*3 rotation matrix t : 3*1 translation vector p p (Essential matrix ) p p ( ) TTyxyx)1,','(')1,,( with 0)](['ppRptp RtEEpp'Rptp with 0)](['E (Essential matrix ) TTyxyx)1,','(')1,,( with 0)](['ppRptp RtEEpp' with 0 babba 000121323aaaaaa rank(E)=2 E t R x x p x=Kp , x =K p p=K-1x, p =K -1x (K,K are the camera calibration matrix ) : p TEp=0 (K -1x )TEK-1x=0 x TK -TEK-1x=0 x TFx=0 F= K -TEK-1 F K E Fundamental matrix F 1 x TFx=0 x=e (e is epipole).

2 X TFe=0, x ( epipolar lines epipole ) Fe = 0 x =e (e is epipole) : e TFx =0, x e TF = 0 FTe = 0 e e TF T from Hartley& Zissermanepipolar pencil x l l = Fx F 2linepoint point-on-line l TFx= 0 l TFx= (Fx)T l = 0 F 3x3 , rank 2 det(F) =0 ( F 7:There are 9elements, but scaling is not significant(-1) and det(F) =0 (-1) ( ) (I)3 ( ) (II) 110000100000000100001333231232221131211w orldworldworldzyxyyxximageimageZYXtrrrtr rrtrrrffokokyxh 11343332312423222114131211worldworldworl dimageimageZYXccccccccccccyxh (3*4 ) (X,Y,Z) (x,y) h XYZxyfhxy1 c11c12c13c14c21c22c23c24c31c32c331 XYZ1 hx c11X c12Y c13Z c14hy c21X c22Y c23Z c24h c31X c32Y c33Z 1 c31Xx c32Yx c33Zx x c11X c12Y c13Z c14c31Xy c32Yy c33Zy y c21X c22Y c23Z c24 3 11343332312423222114131211worldworldworl dimageimageZYXccccccccccccyxh Z=0 Homography D ( D) ( ) 3x3 4 (n>=4))

3 XYZxyf hxy1 c11c12c13c14c21c22c23c24c31c32c33c34 XYZ1 hxy1 c11c12c14c21c22c24c31c32c34 XY1 2n n: 4 Bc=x 4 nnnnnnnnnnnnnnyxyxccccccccyYyXYXxYxXYXyY yXYXxYxXYX 1132312422211412111111111111111000000110 000001xBBBcTT1)( 2minxBc Homography OO111122 HxxHHx D 1H2H x1x24 111122 HxxHHx Homography OOH D to H Homographies for Bird s-eye Viewsfrom Hartley & ZissermanHomographies for Mosaicingfrom Hartley & ZissermanHomographies for Mosaicing 5 Applying Homographies to Removing projective distortion33323113121131'''hyhxhhyhxhxxx 33323123222132'''hyhxhhyhxhxxy 131211333231'hyhxhhyhxhx 232221333231'hyhxhhyhxhy selectfour points in a plane with know coordinates(linear in hij)

4 Single Camera Calibration World Coordinate SystemCamera Coordinate SystemImage Coordinate Systemf:focal lengthkx,ky:scale of the pixel coordinate axiso ,o :image principal pointExtrinsic ParametersIntrinsic ParametersR:rotation matrixt:translation vectorxyTsai s MethodProjective Camera matrix P K 3D-2D Projective mappingProjection matrix (3x4)29 1. 2. 3. P H k, F,E 6 Example Calibration PatternCalibration Pattern: Object with features of known size/geometry Harris or Canny Corner Detector fZXOyxoo,center imageyxkk, size pixelflength focal21, distortion lenskk Calibration = Determine the intrinsic parameters of a cameraxxoZXkfx yyoZYkfy ZXfx ZYfy 10100000010100000000100001cccyyxxcccyyxx imageimageZYXofkofkZYXffokokyx K 10100000010100000000100001 ZYXofkofkZYXffokokyxyyxxyyxximageimage K tx, ty, tz r1, ,3 E 7 1)

5 43(1worldworldworldimageimageZYXyxP 110000100000013,32,31,33,22,21,23,12,11, 1worldworldworldzyxyyxximageimageZYXtrrr trrrtrrrofkofkyx 1010000001cameracameracamerayyxximageima geZYXofkofkyx P P 11rank 3 1)43(1worldworldworldimageimageZYXyxPPXx P 6 points algorithm P 2n 12 n (n 6) 2n P p=0 p=0 p W p W 86 points algorithm TTT( )TT P P==M=KRK: P 3 3 M M=KRK R QR K R K P K R t 10000yyxxofkofk P 3 2 P Tsai EasyCalib OpenCV 9 EasyCalib State-of-the-art calibration Z.)

6 Zhang: Flexible Camera Calibration By Viewing a Plane From Unknown Orientations (1999) Solves correspondence problem Works with planar calibration pad Works well in practiceCalibration Software: OpenCVCalibration Software: Matlab P from Hartley& Zissermanbaseline F F x TFx=0F P x TFx=010 K F E e e OO e t ( ) e R from Hartley& Zissermanepipolar pencil H corner (disparity) B F uncalibrated stereo 11 Tsai Tsai CCD Tsai R T6 f 1 k 1 Tsai Tsai Zhang EasyCalib Tsai OpenCV 12 Homography P F