IEEE TRANSACTIONS ON ROBOTICS, VOL. 32, NO. 6, …

1 IEEE TRANSACTIONS ON ROBOTICS, VOL. 32, NO. 6, DECEMBER 20161309 Past, Present, and Future of SimultaneousLocalization and Mapping: Toward theRobust-Perception AgeCesar Cadena, Luca Carlone, Henry Carrillo, Yasir Latif, Davide Scaramuzza, Jos e Neira, Ian Reid,and John J. LeonardAbstract simultaneous localization and mapping (SLAM) con-sists in the concurrent construction of a model of the environment(themap), and the estimation of the state of the robot movingwithin it. The SLAM community has made astonishing progressover the last 30 years, enabling large-scale real-world applicationsand witnessing a steady transition of this technology to survey the current state of SLAM and consider future direc-tions. We start by presenting what is now thede-factostandardformulation for SLAM. We then review related work, covering abroad set of topics including robustness and scalability in long-termmapping, metric and semantic representations for mapping, the-oretical performance guarantees, active SLAM and exploration,and other new frontiers.

2 This paper simultaneously serves as aposition paper and tutorial to those who are users of SLAM. Bylooking at the published research with a critical eye, we delineateopen challenges and new research issues, that still deserve carefulscientific investigation. The paper also contains the authors takeon two questions that often animate discussions during roboticsconferences:Do robots need SLAM?andIs SLAM solved?Index Terms Factor graphs, localization , mapping, maximuma posteriori estimation, perception, robots, sensing, simultaneouslocalization and mapping (SLAM).Manuscript received July 29, 2016; revised October 31, 2016; accepted Oc-tober 31, 2016. Date of current version December 2, 2016. This paper wasrecommended for publication by Associate Editor J. M. Porta and Editor upon evaluation of the reviewers comments. This work was supportedin part by the following: Grant MINECO-FEDER DPI2015-68905-P, GrantGrupo DGA T04-FSE; ARC Grants DP130104413, Grant CE140100016 andGrant FL130100102; NCCR Robotics; Grant PUJ 6601; EU-FP7-ICT-ProjectTRADR 609763, Grant EU-H2020-688652, and Grant This pa-per was presented in part at the workshop The problem of mobile sensors:Setting future goals and indicators of progress for SLAM at the Robotics: Sci-ence and System Conference, Rome, Italy, July 2015.

3 Additional material forthis paper, including an extended list of references (bibtex) and a table of pointersto online datasets for SLAM, can be found at Cadena is with the Autonomous Systems Laboratory, ETH Z urich, Z urich8092, Switzerland (e-mail: Carlone is with the Laboratory for Information and Decision Systems,Massachusetts Institute of Technology, Cambridge, MA 02139 USA Carrillo is with the Escuela de Ciencias Exactas e Ingenier a, Universi-dad Sergio Arboleda, Bogot a, Colombia, and Pontificia Universidad Javeriana,Bogot a, Colombia (e-mail: Latif and I. Reid are with the School of Computer Science, University ofAdelaide, Adelaide, SA 5005, Australia, and the Australian Center for RoboticVision, Brisbane, QLD 4000, Australia (e-mail: Scaramuzza is with the Robotics and Perception Group, University ofZ urich, Z urich 8006, Switzerland (e-mail: Neira is with the Departamento de Inform atica e Ingenier a de Sistemas,Universidad de Zaragoza, Zaragoza 50029, Spain (e-mail: J.)))))

4 Leonard is with the Marine Robotics Group, Massachusetts Institute ofTechnology, Cambridge, MA 02139 USA (e-mail: versions of one or more of the figures in this paper are available onlineat Object Identifier INTRODUCTIONSLAM comprises the simultaneous estimation of the state ofa robot equipped with on-board sensors and the construc-tion of a model (themap) of the environment that the sensorsare perceiving. In simple instances, the robot state is describedby its pose (position and orientation), although other quantitiesmay be included in the state, such as robot velocity, sensor bi-ases, and calibration parameters. The map, on the other hand,is a representation of aspects of interest ( , position of land-marks, obstacles) describing the environment in which the need to use a map of the environment is twofold. First, themap is often required to support other tasks; for instance, a mapcan inform path planning or provide an intuitive visualizationfor a human operator.)

5 Second, the map allows limiting the errorcommitted in estimating the state of the robot. In the absence of amap, dead-reckoning would quickly drift over time; on the otherhand, using a map, , a set of distinguishable landmarks, therobot can reset its localization error by revisiting known areas(so-calledloop closure). Therefore, SLAM finds applications inall scenarios in which a prior map is not available and needs tobe some robotics applications, the location of a set of land-marks is knownapriori. For instance, a robot operating ona factory floor can be provided with a manually built map ofartificial beacons in the environment. Another example is thecase in which the robot has access to GPS (the GPS satellitescan be considered as moving beacons at known locations). Insuch scenarios, SLAM may not be required if localization canbe done reliably with respect to the known popularity of the SLAM problem is connected with theemergence of indoor applications of mobile robotics.

6 Indooroperation rules out the use of GPS to bound the localizationerror; furthermore, SLAM provides an appealing alternative touser-built maps, showing that robot operation is possible in theabsence of an ad hoc localization thorough historical review of the first 20 years of the SLAM problem is given by Durrant Whyte and Bailey in two sur-veys [8], [70]. These mainly cover what we call theclassical age(1986 2004); the classical age saw the introduction of the mainprobabilistic formulations for SLAM, including approachesbased on extended Kalman filters (EKF), Rao Blackwellizedparticle filters, and maximum likelihood estimation; moreover,it delineated the basic challenges connected to efficiency androbust data association. Two other excellent references describ-ing the three main SLAM formulations of the classical age are1552-3098 2016 IEEE. Personal use is permitted, but republication/redistribution requires IEEE for more TRANSACTIONS ON ROBOTICS, VOL.

7 32, NO. 6, DECEMBER 2016TA B L E ISURVEYING THESURVEYS ANDTUTORIALSYearTopicReference2006 Probabilistic approaches and dataassociationDurrant Whyte and Bailey [8], [70]2008 Filtering approachesAulinaset al.[7]2011 SLAM back endGrisettiet al.[98]2011 Observability, consistency andconvergenceDissanayakeet al.[65]2012 Visual odometryScaramuzza and Fraundofer [86],[218]2016 Multi robot SLAMS aeediet al.[216]2016 Visual place recognitionLowryet al.[160]2016 SLAM in the Handbook of RoboticsStachnisset al.[234, Ch. 46]2016 Theoretical aspectsHuang and Dissanayake [110]the book of Thrun, Burgard, and Fox [240] and the chapter ofStachnisset al.[234, Ch. 46]. The subsequent period is what wecall thealgorithmic-analysis age(2004 2015), and is partiallycovered by Dissanayakeet [65]. The algorithmic analy-sis period saw the study of fundamental properties of SLAM,including observability, convergence, and consistency. In thisperiod, the key role of sparsity toward efficient SLAM solverswas also understood, and the main open-source SLAM librarieswere review the main SLAM surveys to date in Table I, ob-serving that most recent surveys only cover specific aspects orsubfields of SLAM.

8 The popularity of SLAM in the last 30years is not surprising if one thinks about the manifold aspectsthat SLAM involves. At the lower level (called thefront endin Section II), SLAM naturally intersects other research fieldssuch as computer vision and signal processing; at the higherlevel (that we later call theback end), SLAM is an appealingmix of geometry, graph theory, optimization, and probabilisticestimation. Finally, a SLAM expert has to deal with practicalaspects ranging from sensor calibration to system paper gives a broad overview of the current state ofSLAM, and offers the perspective of part of the community onthe open problems and future directions for the SLAM main focus is on metric and semantic SLAM, and we referthe reader to the recent survey by Lowryet al.[160], which pro-vides a comprehensive review of vision-based place recognitionand topological delving into this paper, we first discuss two questionsthat often animate discussions during robotics conferences: doautonomous robots need SLAM?

9 And is SLAM solved as anacademic research endeavor? We will revisit these questions atthe end of the the question Do autonomous robots really needSLAM? requires understanding what makes SLAM aims at building a globally consistent representation ofthe environment, leveraging both ego-motion measurements andloop closures. The keyword here is loop closure : if we sacri-fice loop closures, SLAM reduces to odometry. In early appli-cations, odometry was obtained by integrating wheel pose estimate obtained from wheel odometry quickly drifts,making the estimate unusable after few meters [128, Ch. 6]; thiswas one of the main thrusts behind the development of SLAM:the observation of external landmarks is useful to reduce thetrajectory drift and possibly correct it [185]. However, moreFig. : map built from odometry. The map is homotopic to a long corridorthat goes from the starting position A to the final position B. Points that are closein reality ( , B and C) may be arbitrarily far in the odometric map.

10 Right: mapbuild from SLAM. By leveraging loop closures, SLAM estimates the actualtopology of the environment, and discovers shortcuts in the odometry algorithms are based on visual and inertial in-formation, and have very small drift (< the trajectorylength [83]). Hence the question becomes legitimate: do wereally need SLAM? Our answer is of all, we observe that the SLAM research done over thelast decade has itself produced the visual-inertial odometry al-gorithms that currently represent the state-of-the-art, , [163],[175]; in this sense, visual-inertial navigation (VIN)isSLAM:VIN can be considered areducedSLAM system, in which theloop closure (or place recognition) module is disabled. Moregenerally, SLAM has directly led to the study of sensor fusionunder more challenging setups ( , no GPS, low quality sen-sors) than previously considered in other literature ( , inertialnavigation in aerospace engineering).The second answer regards the true topology of the envi-ronment.