Logo U. Osnabrueck Institute of Computer Science — Knowledge-Based Systems Research Group

Ground Truth Evaluation of Large Urban 6D SLAM

by Oliver Wulf, Andreas Nüchter, Joachim Hertzberg and Bernardo Wagner

Ground Truth Evaluation of Large Urban 6D SLAM


In the past many solutions for simultaneous localization and mapping (SLAM) have been presented. Recently this solutions have been extended to large maps and six degrees of freedom. To demonstrate the capabilities of these SLAM algorithms it is common practice to present the generated maps and successful loop closing. Unfortunately there is often no objective performance metric that allows to compare different approaches. This fact is attributed to the lack of ground truth data. For this reason we present a novel method that is able to generate this ground truth data based on reference maps. Further on the resulting reference path is used to measure the absolute performance of a 6D SLAM algorithm building a large urban outdoor map.

The Video

The video shows the comparison of four different mapping methods. Based on the gyrodometry estimated postions, we use
  1. ICP scan matching, between two consequtive 3D scans (pairwise matching,
  2. ICP scan matching, between all previously acuired scans and the current one (metascan matching),
  3. method (1) with closed loop detection and global relaxation,
  4. method (2) with closed loop detection and global relaxation,

Video (DiVX 71 MB)
Video (low-res MPG 5 MB)

Please use a DivX player to show the video.

The 3D Maps

Here are the four final maps (click to enlarge):
final map, pairwise icp final map, metascan icp final map, pairwise lum final map, metascan lum

References


The Overview Video

This video shows the scanning system in a brief overview.
Video (DiVX 65 MB)
Video (MPG 50 MB)
Andreas Nüchter, March 2007