<!-- TITLE -->
# **Conservative Fine-Tuning of Diffusion Models from Offline Data**

<!-- ## Code -->

## Installation  

Create a conda environment with the following command:

```bash
conda create -n BRAID python=3.10
conda activate BRAID
pip install -r requirements.txt
```

Please use accelerate==0.17.0, other library dependancies might be flexible.

## Training

HuggingFace Accelerate will automatically handle parallel training.  
We conduct our experiments on image tasks using 4 A100 GPUs. Please adjust [*config.train.batch_size_per_gpu_available*] variable in config files according to your GPU memory.  

#### Running STRL (Baseline)  

```bash
accelerate launch offline_main.py --config config/offline.py:aesthetic --config.train.kl_weight 0.01 --config.train.pessimism none
```

#### Running BRAID-Boot

```bash
accelerate launch offline_main.py --config config/offline.py:aesthetic --config.train.kl_weight 0.01 --config.train.pessimism bootstrap
```

#### Running BRAID-Bonus 

```bash
accelerate launch offline_main.py --config config/offline_LCB.py:aesthetic --config.train.kl_weight 0.01
```

## Evaluation

#### Evaluate STRL (Baseline)

```bash
accelerate launch offline_main.py --config config/offline.py:evaluate --config.train.kl_weight 0.01  --config.train.pessimism none --config.resume_from $checkpoint_path
```  

#### Evaluate BRAID-Boot

```bash
accelerate launch offline_main.py --config config/offline.py:evaluate --config.train.kl_weight 0.01 --config.train.pessimism bootstrap --config.resume_from $checkpoint_path
```  

#### Evaluate BRAID-Bonus

```bash
accelerate launch offline_main.py --config config/offline_LCB.py:evaluate --config.train.kl_weight 0.01 --config.train.psm_alpha 1e-3 --config.resume_from $checkpoint_path
```  