Kinect等の色距離センサを用いた 点群処理と3D物体認識

1 Kinect 3D 2016/06/089:45-11:15 22 UbuntuPC .. Desktop-Full 2. 453MB 2008 3 2008 4 2010 3 2010 4 2013 3 2013 4 2013 12 2016 4 person, chair 20 3D Selective Search [Kanezaki+, 2015] G gle RGBD 3D RGB Depth D Kinect RGBD (1)3D DynamicFusion: Reconstruction and Tracking of Non-rigid Scenes in Newcombe, D. Fox, and S. Seitz, CVPR 2015 Best (2)SLAME lasticFusion: Dense SLAM Without A Pose Graph. T. Whelan, S. Leutenegger, B. Glocker, R.

2 F. Salas-Moreno, and A. Davison, Robotics: Science and Systems (RSS), (3) Structured Indoor Modeling. S. Ikehata, H. Yan, and Y. Furukawa, ICCV 2015 (oral) (4) holoportation: virtual 3D teleportation in real-time(Microsoft Research) 1. RGBD 3D RGBD 2. 3D Point Cloud Library (PCL) 3. 3D 4. 3D 3D 2D PCL 3D Model Retrieval Using Probability Density-Based Shape Descriptors. C. B. Akg l, B. Sankur, Y. Yemez, and F. Schmitt, PAMI, 20093D 2D Kazhdan, M., Funkhouser, T., & Rusinkiewicz, S. Rotation invariant spherical harmonic representation of 3D shape descriptors. In Proceedings of the Eurographics/ACM SIGGRAPH symposium on Geometry processing(pp. 156-164), Harmonic Representations 0),(),(lllmmlmlYcf 3D 2D Ding-Yun Chen, Xiao-Pei Tian, Yu-TeShen and Ming Ouhyoung, "On Visual Similarity Based 3D Model Retrieval",Computer Graphics Forum (EUROGRAPHICS'03), Vol.

3 22, No. 3, pp. 223-232, Sept. Field Descriptor3D ~dynamic/3D 2D Ankerst, M., Kastenm ller, G., Kriegel, H. P., & Seidl, T. (1999, January). 3D shape histograms for similarity search and classification in spatial databases. InAdvances in Spatial Databases(pp. 207-226), Shape Histograms3D Johnson, Andrew E., and Martial Hebert. "Using spin images for efficient object recognition in cluttered 3D scenes."Pattern Analysis and Machine Intelligence, IEEE Transactions (1999): Image 2D 3D R. B. Rusu, N. Blodow, Z. C. Marton, and M. Beetz, Aligning point cloud views using persistent feature histograms, in Proc. Int. Conf. Intelligent Robots and Systems (IROS), )),)(arctan((/)()(iiiiiiinnnnpnppnnnp ip nin , , ( 2) : Point Feature Histogram 2D 3D R. B. Rusu, N.

4 Blodow, and M. Beetz, Fast point feature histograms (FPFH) for 3-D registration, in Proc. Int. Conf. Robotics and Automation (ICRA), )),)(arctan((/)()(iiiiiiinnnnpnppnnnp ip ninSPFH , , ( ) : kiiipkpp1)(SPFH11)(SPFH)(FPFH Fast Point Feature Histogram 2D 3D R. B. Rusu, G. Bradski, R. Thibaux, and J. Hsu, Fast 3-D recognitionand pose using the viewpoint feature histogram, in Proc. Int. Conf. Intelligent Robots and Systems (IROS), Feature Histogram 2D Global FPFH Extended FPFH 3D A. Aldoma, N. Blodow, D. Gossow, S. Gedikli, R. Rusu, M. Vincze, and G. Bradski, CAD-model recognition and 6 DOF pose estimation using 3D cues, in Proc. ICCV workshop on 3dRR, 2011 CVFH 2D Clustered Viewpoint Feature HistogramCVFHCVFHCVFH(1) (2) VFH (3) Shape Distribution Component (SDC)

5 VFH VFHSDCCVFH3D , , ,and , OUR-CVFH Oriented, Unique and Repeatable Clustered Viewpoint Feature Histogram for Object Recognition and 6 DOF Pose Estimation , in Joint DAGM-OAGM Pattern Recognition Symposium, 2D Oriented, Unique and RepeatableClustered Viewpoint Feature HistogramCVFHSGURF viewpointcomponentSGURF SGURF OUR-CVFHSGURFVFHSDCCVFHOUR-CVFH RGBD 083875142211050100150200250 RGBD RGBD 2008 ICRA2011 MESA SR-4000 TOF sensorPointGrayFlea2 camera176 144 pixel 100 ICRA2011 2010 9 2010 2011 2012 2013 2014 2015 2010 11 Kinect 640 480 pixelRGBD ROS3D Contest ICRA2011 Best Vision Paper:Sparse Distance Learning for Object Recognition Combining RGB and Depth Information Kevin Lai, LiefengBo, XiaofengRen, and Dieter Fox RGBD :1st Place ($3000):Customizable Buttons2nd Place ($2000):Quadrotor Altitude and Obstacle Avoidance3rd Place ($1000):Humanoid Teleoperation4th Place ($500):Person Tracking and Reconstruction from a Mobile Base with a 7 DOF ManipulatorMost Useful:1st Place ($2000):RGBD-6D-SLAM2nd Place ($1000):Automatic Calibration of Extrinsic ParametersRGBD 2 3 RGB RGBD VisionRobotGraphicsD Partialdata RGBD 1/2 Multimodal templates for real-time detection of texture-less objects in heavily cluttered scenesStefan Hinterstoisser, Stefan Holzer, Cedric Cagniart, Slobodan Ilic, Kurt Konolige, Nassir Navab, and Vincent ICCV, 2011.

6 In in can parse a VGAimage with over 3000 templateswith about 10 fpson the CPU RGBD 2/2 MMSS: Multi-modal Sharable and Specific Feature Learning forRGB-D Object RecognitionAnranWang, JianfeiCai, JiwenLu, and Tat-Jen Cham. IEEE ICCV, feature learningpre-training fully-connected RGB Depth Deep CNN RGBD Combining color and shape descriptors for 3D model al., Signal Processing: Image Communication , 2013. 1/2 Color Spin Image Spin Image Spin Image ( +1) Spin Image PCA Quantization level: 1 2 3 .. RGBD Unique signatures of histograms for local surface description[Tombari et al., ECCV2010] SHOT 3D PCL 8 x 2 x 2 = A combined texture-shape descriptor for enhanced 3D feature matching[Tombariet al.]

7 , ICIP2011] CSHOT SHOT 2/2 RGBD The Partial View Heat Kernel Descriptor for 3D Object Representation[Brandaoet al., ICRA2014] Heat Kernel Signature (HKS) Partial View cf.) Heat Kernel Signature (HKS) non-rigid rigid HKS , , = =1 , , Laplace-Beltrami partial view RGBD Learning Similarities for Rigid and Non-Rigid Object Detection[Kanezakiet al., 3DV2014] QAP RGBD (a) Initial correspondences (b) Selected correspondences (c) Selected correspondencestrue positivefalse positivetrue positivefalse positive UbuntuPC .. Desktop-Full 2. 453MB 1. 3D 3D 3D 3D 3D SketchUp 3D 3D 3D OBJ (*.

8 Obj) 3D MeshLab & 3D OBJ vertex face / / 3D OBJ vertex face / / Point Cloud 3D PointCloud PCL .pcd # .PCD -Point Cloud Data file formatVERSION x y z rgbSIZE 4 4 4 4 TYPE F FFFCOUNT 1 1 1 1 WIDTH 640 HEIGHT 480 VIEWPOINT 0 0 0 1 0 0 0 POINTS 307200 DATA 0 + 0 + 0 + 0 + 0 + 0 + 0 + 0 + 0 + 0 + 0 + 0 + 0 + 0 + 0 + 0 +06 ASCII BINARY Kinect RGBD ROS OS Kinectv1 C++ 5. Kinect ROS Ubuntu Kinectv2 v2 ASUS Xtion OK Python Kinect RGBD 5.

9 Kinect $ sudosh c echo deb $(lsb_release-sc) main > /etc/ $ sudoapt-key adv-- :80 --recv-key 0xB01FA116$ sudoapt-get update$ sudoapt-get install ros-jade-desktop-full$ sudorosdepinit$ rosdepupdate$ echo "source /opt/ros/ " >> ~/.bashrc$ source ~/.bashrc# jade ROS # cturtle, diamondback, electric, fuerte, groovy, hydro, indigo, jade .bashrc $ source /opt/ros/ Kinect RGBD 5. Kinect Kinect ROS $ sudoapt-get installros-jade-openni-launch OpenNI $ sudoapt-get installros-indigo-freenect-launch# jade freenect_launch indigo # ROS # Ubuntu - ROS _ $ sudoapt-get installros-jade-openni2-launchXtion Kinect RGBD 5.

10 Kinect Kinect $ roslaunchopenni2_launch $ source /opt/ros/ $ or Xtion Kinect RGBD 5. Kinect Kinect $ roslaunchopenni2_launch $ source /opt/ros/ $ or Xtion $ roscore $ rosbagplay or $ 453MB Kinect RGBD 5. Kinect ROS $ rostopiclist ROS publish $ rostopichz/camera/rgb/image_color Kinect RGBD 5. Kinect ROS $ rostopiclist ROS publish $ rostopichz/camera/rgb/image_color Xtion ROS /camera/rgb/image_raw image_color image_raw Kinect RGBD 5.