Paris-CARLA-3D: a Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D Mapping

Paris Point Cloud RGB Paris Point Cloud Semantic

Password is: Paris-CARLA-3D

Paris-CARLA-3D is a dataset of several dense colored point clouds of outdoor environments built by a mobile LiDAR and camera system. The data are composed of two sets with synthetic data from the open source CARLA simulator (700 million points) and real data acquired in the city of Paris (60 million points), hence the name Paris-CARLA-3D.

One of the advantages of this dataset is to have simulated the same LiDAR and camera platform in the open source CARLA simulator as the one used to produce the real data. In addition, manual annotation of the classes using the semantic tags of CARLA was performed on the real data, allowing the testing of transfer methods from the synthetic to the real data.

The objective of this dataset is to provide a challenging dataset to evaluate and improve methods on difficult vision tasks for the 3D mapping of outdoor environments: semantic segmentation, instance segmentation, and scene completion.

The Python scripts to generate the synthetic data in CARLA v0.9.10: https://github.com/jedeschaud/paris_carla_simulator

You can find more details in the following article :

Deschaud J.-E., Duque D., Richa J. P., Velasco-Forero S., Marcotegui B., Goulette F., Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D

If you use this dataset, do not forget to cite our article:

@article{deschaud2021pariscarla3d,
  author = {Deschaud, Jean-Emmanuel and Duque, David and Richa, Jean Pierre and Velasco-Forero, Santiago and Marcotegui, Beatriz and Goulette, François},
  title = {Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D Mapping},
  journal = {Remote Sensing},
  volume = {13},
  year = {2021},
  number = {22},
  url = {https://www.mdpi.com/2072-4292/13/22/4713},
  issn = {2072-4292},
  doi = {10.3390/rs13224713}
}

The different benchmarks are presented below. If you want your results on Paris-CARLA-3D to be published on that page, please send an email to jean-emmanuel.deschaud(at)mines-paristech.fr with a link to your paper (it can be an arXiv paper)

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Semantic Segmentation Task

Supervised Semantic Segmentation on Paris data (annotations from CARLA and Paris training data can be used)

Methods	Overall mIoU	S0 mIoU	S3 mIoU	Reference
KPConv	51.7	45.2	62.9	KPConv: Flexible and Deformable Convolution for Point Clouds, https://arxiv.org/pdf/1904.08889.pdf
PointNet++	19.9	13.9	25.8	PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space, https://arxiv.org/pdf/1706.02413.pdf

mIoU is the mean Intersection over Union, S0 = Soufflot0, S3 = Soufflot3 (all results are in %)

Unsupervised domain adaptation from CARLA to Paris data (annotations from Paris training data cannot be used, only annotations from CARLA training data)

Methods	Overall mIoU	S0 mIoU	S3 mIoU	Reference
KPConv (Source only)	19.2	20.6	17.7	KPConv: Flexible and Deformable Convolution for Point Clouds, https://arxiv.org/pdf/1904.08889.pdf

mIoU is the mean Intersection over Union, S0 = Soufflot0, S3 = Soufflot3 (all results are in %).

Instance Segmentation Task

Supervised Instance Segmentation on CARLA and Paris data (annotations from CARLA and Paris training data can be used)

Methods	SM	PQ	mIoU	Reference
KPconv + Mathematical Morphology	55.3	38.4	44.2	Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D, https://www.mdpi.com/2072-4292/13/22/4713

SM = Segment Match, PQ = Panoptic Quality, mIoU = mean Interstection over Union (all results are in %).

Scene Completion Task

Self-supervised completion of point cloud chunks (128x128x128 with voxel size of 5 cm) on Paris data (all training from CARLA and Paris data can be used)

Methods	CD	Reference
SG-NN with Hoppe TSDF	7.5 cm	Paris-CARLA-3D: A Real and Synthetic Outdoor Point Cloud Dataset for Challenging Tasks in 3D, https://www.mdpi.com/2072-4292/13/22/4713

CD = Chamfer Distance between original and predicted point clouds

Completion of full point cloud scene with ground truth model - To appear soon

Novel View Synthesis Task

To appear soon

Paris-CARLA-3D dataset is made available under the Creative Commons Attribution Non-Commercial No Derivatives (CC-BY-NC-ND-3.0) License.