# Point-GT

## Geodesic Self-Attention for 3D Point Clouds


In this work, we introduce a novel metric on the Riemannian manifold to capture the long-range geometrical dependencies of point cloud objects to replace traditional self-attention modules, namely, the Geodesic Self-Attention module.

## 1. Requirements
PyTorch >= 1.7.0;
python >= 3.7;
CUDA >= 9.0;
GCC >= 4.9;
torchvision;

```
pip install -r requirements.txt
```

```
# PointNet++
pip install "git+git://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops&subdirectory=pointnet2_ops_lib"
# GPU kNN
pip install --upgrade https://github.com/unlimblue/KNN_CUDA/releases/download/0.2/KNN_CUDA-0.2-py3-none-any.whl
```

## 2. Datasets

We use ShapeNet, ScanObjectNN, ModelNet40 and ShapeNetPart, SHREC 15 in this work. See [DATASET.md](./DATASET.md) for details.

## 3. Point-GT Models

The pretrained model can be downloaded [here](https://drive.google.com/file/d/1XGl9s5EktDsRAO_AmPXZm3uyN6OTRKnC/view?usp=sharing).

| Task                            | Dataset | Config | Acc.       | Download|      
|---------------------------------| ----- |-----|------------| -----|
| Classification (Point-GT-MAE)   | ScanObjectNN |[finetune_scan_hardest.yaml](./cfgs/finetune_scan_hardest.yaml)| 85.7%      | [here](https://drive.google.com/file/d/17-ghLxODcd22uAAAr-Os3dqgrLKIHpCU/view?usp=sharing)  |
| Classification (Point-GT-MAE)   | ScanObjectNN |[finetune_scan_objbg.yaml](./cfgs/finetune_scan_objbg.yaml)| 89.5%      | [here](https://drive.google.com/file/d/12g7Iw7MhLTRWaCgJdWldjw-FCnvDIFjB/view?usp=sharing) |
| Classification (Point-GT-MAE)   | ScanObjectNN |[finetune_scan_objonly.yaml](./cfgs/finetune_scan_objonly.yaml)| 90.7%      | [here](https://drive.google.com/file/d/1wWptqoC8stk-M1_WOxRU6I-t3hABn30Z/view?usp=sharing) |
| Classification (Point-GT-MAE)   | ModelNet40(8k) |[finetune_modelnet_8k.yaml](./cfgs/finetune_modelnet_8k.yaml)| 94.1%      | [here](https://drive.google.com/file/d/1Czx3yNbTpC0yWyL3txNDOh0kP1-P11t4/view?usp=sharing) |
| Part segmentation (Point-GT-G)  | ShapeNetPart| [segmentation](./segmentation)| 85.9% mIoU | [here](https://drive.google.com/file/d/1K5IrL_LSdENXLjV4aR9__mMLaMEz5YKk/view?usp=sharing) |
| Part segmentation (Point-GT-DM) | ShapeNetPart| [segmentation](./segmentation)| 85.8% mIoU | [here](https://drive.google.com/file/d/1XztcPxOAT5I7p_KsQX3Aq5gTdN5gI5fX/view?usp=sharing) |

| Task                                 | Dataset | Config | 5w10s Acc. (%) | 5w20s Acc. (%) | 10w10s Acc. (%) | 10w20s Acc. (%) |     
|--------------------------------------| ----- |-----|----------------|----------------|-----------------|-----------------|
| Few-shot learning (Point-GT-G-rand)  | ModelNet40 |[fewshot.yaml](./cfgs/fewshot.yaml)| 95.4 ± 2.5     | 96.7 ± 2.4     | 90.2 ± 5.4      | 93.0 ± 4.4      |
| Few-shot learning (Point-GT-DM-rand) | ModelNet40 |[fewshot.yaml](./cfgs/fewshot.yaml)| 93.1 ± 3.9     | 96.1 ± 3.4     | 90.1 ± 5.6      | 92.5 ± 4.9      |
| Few-shot learning (Point-GT-MAE)     | ModelNet40 |[fewshot.yaml](./cfgs/fewshot.yaml)| 96.3 ± 2.3     | 98.2 ± 1.5     | 92.8 ± 4.6      | 95.1 ± 3.6      |


## 4. Point-GT Training

Before running the classification code, rename the model you want to test in ./models/(Point_GT_G.py, Point_GT_DM.py) to Point_GT.py.

Training from scratch on ScanObjectNN, run:
```
CUDA_VISIBLE_DEVICES=<GPUs> python main.py --config cfgs/finetune_scan_hardest.yaml \
--finetune_model --exp_name <output_file_name> --ckpts <path/to/pre-trained/model>
```
Fine-tuning on ScanObjectNN, run:
```
CUDA_VISIBLE_DEVICES=<GPUs> python main.py --config cfgs/finetune_scan_hardest.yaml \
--finetune_model --exp_name <output_file_name> --ckpts <path/to/pre-trained/model>
```
Training from scratch on ModelNet40, run:
```
CUDA_VISIBLE_DEVICES=<GPUs> python main.py --config cfgs/finetune_modelnet.yaml \
--finetune_model --exp_name <output_file_name>
```
Fine-tuning on ModelNet40, run:
```
CUDA_VISIBLE_DEVICES=<GPUs> python main.py --config cfgs/finetune_modelnet.yaml \
--finetune_model --exp_name <output_file_name>
```
Voting on ModelNet40, run:
```
CUDA_VISIBLE_DEVICES=<GPUs> python main.py --test --config cfgs/finetune_modelnet.yaml \
--exp_name <output_file_name> --ckpts <path/to/best/fine-tuned/model>
```
Few-shot learning, run:
```
CUDA_VISIBLE_DEVICES=<GPUs> python main.py --config cfgs/fewshot.yaml --finetune_model \
--ckpts <path/to/pre-trained/model> --exp_name <output_file_name> --way <5 or 10> --shot <10 or 20> --fold <0-9>
```
Part segmentation on ShapeNetPart, run:
```
cd segmentation
python main.py --model modelname --root path/to/data --learning_rate 0.0002 --epoch 300
```


## Acknowledgements

Our codes are built upon [Geoopt](https://github.com/geoopt/geoopt), [Point-BERT](https://github.com/lulutang0608/Point-BERT), [Point-MAE](https://github.com/Pang-Yatian/Point-MAE), [Pointnet2_PyTorch](https://github.com/erikwijmans/Pointnet2_PyTorch) and [Pointnet_Pointnet2_pytorch](https://github.com/yanx27/Pointnet_Pointnet2_pytorch)

