Paris-CARLA-3D: a Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D Mapping

 

Paris Point Cloud RGB  Paris Point Cloud Semantic

 

Link to download Paris-CARLA-3D dataset

 

Paris-CARLA-3D is a dataset of several dense colored point clouds of outdoor environments built by a mobile LiDAR and camera system. The data are composed of two sets with synthetic data from the open source CARLA simulator (700 million points) and real data acquired in the city of Paris (60 million points), hence the name Paris-CARLA-3D.

 

One of the advantages of this dataset is to have simulated the same LiDAR and camera platform in the open source CARLA simulator as the one used to produce the real data. In addition, manual annotation of the classes using the semantic tags of CARLA was performed on the real data, allowing the testing of transfer methods from the synthetic to the real data.

 

The objective of this dataset is to provide a challenging dataset to evaluate and improve methods on difficult vision tasks for the 3D mapping of outdoor environments: semantic segmentation, instance segmentation, and scene completion.

 

The Python scripts to generate the synthetic data in CARLA v0.9.10: https://github.com/jedeschaud/paris_carla_simulator

 

You can find more details in the following article : 

Deschaud J.-E., Duque D., Richa J. P., Velasco-Forero S., Marcotegui B., Goulette F., Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D

 

If you use this dataset, do not forget to cite our article:

@article{deschaud2021pariscarla3d,
  author = {Deschaud, Jean-Emmanuel and Duque, David and Richa, Jean Pierre and Velasco-Forero, Santiago and Marcotegui, Beatriz and Goulette, François},
  title = {Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D Mapping},
  journal = {Remote Sensing},
  volume = {13},
  year = {2021},
  number = {22},
  url = {https://www.mdpi.com/2072-4292/13/22/4713},
  issn = {2072-4292},
  doi = {10.3390/rs13224713}
}

 

The different benchmarks are presented below. If you want your results on Paris-CARLA-3D to be published on that page, please send an email to jean-emmanuel.deschaud(at)mines-paristech.fr with a link to your paper (it can be an arXiv paper)

 

Semantic Segmentation Task

  • Supervised Semantic Segmentation on Paris data (annotations from CARLA and Paris training data can be used)
Methods Overall mIoU S0 mIoU S3 mIoU Reference
KPConv 51.7 45.2 62.9 KPConv: Flexible and Deformable Convolution for Point Clouds, https://arxiv.org/pdf/1904.08889.pdf
PointNet++ 19.9 13.9 25.8 PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space, https://arxiv.org/pdf/1706.02413.pdf

mIoU is the mean Intersection over Union, S0 = Soufflot0, S3 = Soufflot3 (all results are in %)

 

  • Unsupervised domain adaptation from CARLA to Paris data (annotations from Paris training data cannot be used, only annotations from CARLA training data)
Methods Overall mIoU S0 mIoU S3 mIoU Reference
KPConv 19.2 20.6 17.7 KPConv: Flexible and Deformable Convolution for Point Clouds, https://arxiv.org/pdf/1904.08889.pdf

mIoU is the mean Intersection over Union, S0 = Soufflot0, S3 = Soufflot3 (all results are in %).

 

Instance Segmentation Task

  • Supervised Instance Segmentation on CARLA and Paris data (annotations from CARLA and Paris training data can be used)
Methods SM PQ mIoU Reference
KPconv + Mathematical Morphology 55.3 38.4 44.2 Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D, https://www.mdpi.com/2072-4292/13/22/4713

SM = Segment Match, PQ = Panoptic Quality, mIoU = mean Interstection over Union (all results are in %).

 

Scene Completion Task

  • Self-supervised completion of point cloud chunks (128x128x128 with voxel size of 5 cm) on Paris data (all training from CARLA and Paris data can be used)
Methods CD Reference
SG-NN with Hoppe TSDF 7.5 cm Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D, https://www.mdpi.com/2072-4292/13/22/4713

CD = Chamfer Distance between original and predicted point clouds

 

  • Completion of full point cloud scene with ground truth model - To appear soon

 

Novel View Synthesis Task

To appear soon

 

 

Paris-CARLA-3D dataset is made available under the Creative Commons Attribution Non-Commercial No Derivatives (CC-BY-NC-ND-3.0) License.