3D is here: Point Cloud Library (PCL)

1 3D is here: Point Cloud Library (PCL)Radu Bogdan Rusu and Steve CousinsWillow Garage68 Willow Rd., Menlo Park, CA 94025, With the advent of new, low-cost 3D sensinghardware such as the Kinect, and continued efforts in advancedpoint Cloud processing, 3D perception gains more and moreimportance in robotics, as well as other this paper we present one of our most recent initiatives inthe areas of Point Cloud perception: PCL ( Point Cloud Library ). PCL presents an advancedand extensive approach to the subject of 3D perception, andit s meant to provide support for all the common 3D buildingblocks that applications need. The Library contains state-of-the art algorithms for: filtering, feature estimation, surfacereconstruction, registration, model fitting and is supported by an international community of roboticsand perception researchers. We provide a brief walkthrough ofPCL including its algorithmic capabilities and INTRODUCTIONFor robots to work in unstructured environments, they needto be able to perceive the world.

2 Over the past 20 years,we ve come a long way, from simple range sensors basedon sonar or IR providing a few bytes of information aboutthe world, to ubiquitous cameras to laser scanners. In thepast few years, sensors like the Velodyne spinning LIDAR used in the DARPA Urban Challenge and the tilting laserscanner used on the PR2 have given us high-quality 3 Drepresentations of the world - Point clouds. Unfortunately,these systems are expensive, costing thousands or tens ofthousands of dollars, and therefore out of the reach of manyrobotics recently, however, 3D sensors have become availablethat change the game. For example, the Kinect sensor forthe Microsoft XBox 360 game system, based on underlyingtechnology from PrimeSense, can be purchased for under$150, and provides real time Point clouds as well as 2 Dimages. As a result, we can expect that most robots in thefuture will be able to see the world in 3D.

3 All that sneeded is a mechanism for handling Point clouds efficiently,and that s where the open source Point Cloud Library , PCL,comes in. Figure 1 presents the logo of the is a comprehensive free, BSD licensed, Library forn-D Point Clouds and 3D geometry processing. PCL isfully integrated with ROS, the Robot Operating System ( ), and has been already used in a varietyof projects in the robotics ARCHITECTURE ANDIMPLEMENTATIONPCL is a fully templated, modern C++ Library for 3 Dpoint Cloud processing. Written with efficiency and per-Fig. Point Cloud Library in mind on modern CPUs, the underlying datastructures in PCL make use of SSE optimizations mathematical operations are implemented with andbased on Eigen, an open-source template Library for linearalgebra [1]. In addition, PCL provides support for OpenMP( ) and Intel Threading BuildingBlocks (TBB) Library [2] for multi-core parallelization.

4 Thebackbone for fast k-nearest neighbor search operations isprovided by FLANN (Fast Library for Approximate NearestNeighbors) [3]. All the modules and algorithms in PCL passdata around using Boost shared pointers (see Figure 2), thusavoiding the need to re-copy data that is already presentin the system. As of version , PCL has been ported toWindows, MacOS, and Linux, and Android ports are in an algorithmic perspective, PCL is meant to incor-porate a multitude of 3D processing algorithms that operateon Point Cloud data, including: filtering, feature estimation,surface reconstruction, model fitting, segmentation, registra-tion, etc. Each set of algorithms is defined via base classesthat attempt to integrate all the common functionality usedthroughout the entire pipeline, thus keeping the implementa-tions of the actual algorithms compact and clean. The basicinterface for such a processing pipeline in PCL is: create the processing object ( , filter, feature estima-tor, segmentation); use setInputCloud to pass the input Point Cloud datasetto the processing module; set some parameters; call compute (or filter, segment, etc) to get the sequence of pseudo-code presented in Figure 2 showsa standard feature estimation process in two steps, where aNormalEstimationobject is first created and passed an inputdataset, and the results together with the original input arethen passed together to an FPFH [4] estimation further simplify development, PCL is split into a seriesof smaller code libraries, that can be compiled separately: libpclfilters: implements data filters such as downsam-NormalEstimationPointCloudConstS haredPtr & Cloud [PointCloud &normals]PointCloudConstSharedPtr &normalsFPFHE stimation[PointCloud &fpfh]Fig.

5 Example of the PCL implementation pipeline for Fast PointFeature Histogram (FPFH) [4] , outlier removal, indices extraction, projections,etc; libpclfeatures: implements many 3D features such assurface normals and curvatures, boundary Point estima-tion, moment invariants, principal curvatures, PFH andFPFH descriptors, spin images, integral images, NARF descriptors, RIFT, RSD, VFH, SIFT on intensity data,etc; libpclio: implements I/O operations such as writingto/reading from PCD ( Point Cloud Data) files; libpclsegmentation: implements cluster extraction,model fitting via sample consensus methods for a va-riety of parametric models (planes, cylinders, spheres,lines, etc), polygonal prism extraction, etc libpclsurface: implements surface reconstruction tech-niques, meshing, convex hulls, Moving Least Squares,etc; libpclregistration: implements Point Cloud registrationmethods such as ICP, etc; libpclkeypoints: implements different keypoint extrac-tion methods, that can be used as a preprocessing stepto decide where to extract feature descriptors; libpclrangeimage: implements support for range im-ages created from Point Cloud ensure the correctness of operations in PCL, themethods and classes in each of the above mentioned librariescontain unit and regression tests.

6 The suite of unit tests iscompiled on demand and verified frequently by a dedicatedbuild farm, and the respective authors of a specific compo-nent are being informed immediately when that componentfails to test. This ensures that any changes in the code aretested throughly and any new functionality or modificationwill not break already existing code that depends on addition, a large number of examples and tutorialsare available either as C++ source files, or as step-by-stepinstructions on the PCL wiki web PCLANDROSOne of the corner stones in the PCL design philosophyis represented by Perception Processing Graphs (PPG). Therationality behind PPGs are that most applications that dealwith Point Cloud processing can be formulated as a concreteset of building blocks that are parameterized to achieve dif-ferent results. For example, there is no algorithmic differencebetween a wall detection algorithm, or a door detection, or atable detection all of them share the same building block,which is in this case, a constrained planar segmentationalgorithm.

7 What changes in the above mentioned cases isa subset of the parameters used to run the this in mind, and based on the previous experience ofdesigning other 3D processing libraries, and most recently,ROS, we decided to make each algorithm from PCL availableas a standalone building block, that can be easily connectedwith other blocks, thus creatingprocessing graphs, in thesame way that nodes connect together in a ROS , because Point clouds are extremely large innature, we wanted to guarantee that there would be nounnecessary data copying or serialization/deserialization forcritical applications that can afford to run in the sameprocess. For this we creatednodelets, which are dynamicallyloadable plugins that look and operate likeROS nodes, butin a single process (as single or multiple threads).A concrete nodelet PPG example for the problem ofidentifying a set of Point clusters supported by horizontalplanar areas is shown in Figure 3.

8 A ROS nodelet graph for the problem of object clustering on VISUALIZATIONPCL comes with its own visualization Library , based onVTK [5]. VTK offers great multi-platform support for ren-dering 3D Point Cloud and surface data, including visualiza-tion support for tensors, texturing, and volumetric Visualizationlibrary is meant to integrate PCLwith VTK, by providing a comprehensive visualization layerfor n-D Point Cloud structures. Its purpose is to be ableto quickly prototype and visualize the results of algorithmsoperating on such hyper-dimensional data. As of version ,the visualization Library offers: methods for rendering and setting visual properties(colors, Point sizes, opacity, etc) for any n-D Point clouddataset; methods for drawing basic 3D shapes on screen ( ,cylinders, spheres, lines, polygons, etc) either from setsof points or from parametric equations; a histogram visualization module (PCLH istogramVisu-alizer) for 2D plots; a multitude ofgeometryandcolorhandlers.

9 Here, theuser can specify what dimensions are to be used for thepoint positions in a 3D Cartesian space (see Figure 4),or what colors should be used to render the points (seeFigure 5); RangeImage visualization modules (see Figure 6).Thehandler interactorsare modules that describe howcolors and the 3D geometry at each Point in space arecomputed, displayed on screen, and how the user interactswith the data. They are designed with simplicity in mind,and are easily extendable. A code snippet that producesresults similar to the ones shown in Figure 4 is presentedin Algorithm 1 Code example for the results shown in Figure namespace pclvisualization;PCLV isualizer p ( Test );PointCloudColorHandlerRandom handler ( Cloud ); ( Cloud , handler, cloudrandom ); (); ( cloudrandom );PointCloudGeometryHandlerSurfaceNormal handler2 ( Cloud ); ( Cloud , handler2, cloudrandom ); ();The Library also offers a few general purpose tools forvisualizing PCD files, as well as for visualizing streams ofdata from a sensor in real-time in 4.

10 An example of two differentgeometryhanders applied to the samedataset. Left: the 3D Cartesian space represents XYZ data, with the arrowsrepresenting surface normals estimated at each Point in the Cloud , right: theCartesian space represents the 3 dimensions of the normal vector at eachpoint for the same example of two differentcolorhanders applied to thesame dataset. Left: the colors represent the distance from the acquisitionviewpoint, right: the color represent the RGB texture acquired at each example of a RangeImage display using PCL Visualization(bottom) for a given 3D Point Cloud dataset (top).V. USAGEEXAMPLESIn this section we present two code snippets that exhibitthe flexibility and simplicity of using PCL for filteringand segmentation operations, followed by three applicationexamples that make use of PCL for solving the perceptionproblem: i) navigation and mapping, ii) object recognition,and iii) manipulation and constitutes one of the most important operationsthat any raw Point Cloud dataset usually goes through, beforeany higher level operations are applied to it.