# Rad-NeRF

The official implementation of Paper "Rad-NeRF: Ray-decoupled Training of Neural Radiance Field"

# Installation
## Hardware

* OS: Ubuntu 20.04
* Tested with RTX 3090, CUDA 11.8 (might also work with previous version)

## Software

* Create a new python environment
    ```
    conda create -n rad_nerf python=3.8
    conda activate rad_nerf
    ```  
* Install pytorch and related libraries
    ```
    pip install torch==2.0.0 torchvision==0.15.1 torchaudio==2.0.1 --index-url https://download.pytorch.org/whl/cu118
    ```  
* Install other necessary libiraries
    * Install `torch-scatter` following their [instruction](https://github.com/rusty1s/pytorch_scatter#installation)
    * Install `tinycudann` following their [instruction](https://github.com/NVlabs/tiny-cuda-nn#pytorch-extension) (pytorch extension)
    * Install `apex` following their [instruction](https://github.com/NVIDIA/apex#linux)
    * Install core requirements by `pip install -r requirements.txt`

* Cuda extension: Upgrade `pip` to >= 22.1 and run `pip install models/csrc/` (please run this each time you `pull` the code)

# :books: Supported Datasets

1.  [NSVF](https://github.com/facebookresearch/NSVF#dataset).

2.  [NeRF++](https://github.com/Kai-46/nerfplusplus#data).

3.  [MipNeRF360](http://storage.googleapis.com/gresearch/refraw360/360_v2.zip).

4.  [Free](https://totoro97.github.io/projects/f2-nerf#data)


# Training and evaluation

We provide example scripts for MaskTAT Dataset.(The data is under ./data/TanksAndTemple, including five scenes: Ignatius, Truck, Barn, Family, Caterpillar)

* Instant-NGP Baseline
    ```
    $SCENE_NAME = Ignatius/Truck/Barn/Family/Caterpillar

    CUDA_VISIBLE_DEVICES=0 python train.py \
    --root_dir ./data/TanksAndTemple/$SCENE_NAME --dataset_type nsvf --dataset_name TanksAndTemple --exp_name Instant-NGP \
    --scene_name $SCENE_NAME --downsample 1 \
    --num_epochs 20 --batch_size 8192 --lr 1e-2 --scale 0.5 --eval_lpips
    ```

* Rad-NeRF Exp
    ```
    $SCENE_NAME = Ignatius/Truck/Barn/Family/Caterpillar


    CUDA_VISIBLE_DEVICES=0 python train_ml.py \
    --root_dir ./data/TanksAndTemple/$SCENE_NAME --dataset_type nsvf --dataset_name TanksAndTemple --exp_name Rad-NeRF \
    --scene_name $SCENE_NAME --downsample 1 \
    --num_epochs 20 --batch_size 8192 --lr 1e-2 --scale 0.5 --eval_lpips \
    --moe_training --model_zoo_size 2 --gate_type ray --depth_mutual_loss_w 5e-3 --cv_loss_w 1e-2
    ```

The execution scripts for other datasets are similar with the above examples.

More options can be found in [opt.py](opt.py).
