Professor, ECCAI Fellow
Sapienza Università di Roma, Italy
In this work, we address the issue of benchmarking Semantic Mapping, the ability to build a representation of the environment that combines geometric and symbolic knowledge about the environment. We first analyze the evaluations proposed in theliterature. Then, we address two new possible approaches: benchmarking through competitions and building data sets for off-line evaluation. In the first case, an experimental scenario for the comparison of different systems is designed within a physical testbed inspired by an indoor domestic environment. In the second case, a systematic procedure is defined for building datasets, consisting of the RGBD reconstruction of the testing environment, where semantic ground truth can be annotated in a standardized format. The relevant software for data collection is made available to the community and a systematic approach to performance evaluation is introduced.