# Standard Benchmarks



## Preparing datasets

**CIFAR-10:** Download the [CIFAR-10 python version](https://www.cs.toronto.edu/~kriz/cifar.html) and convert to ZIP archive:

```.bash
python dataset_tool.py --source=downloads/cifar10/cifar-10-python.tar.gz \
    --dest=datasets/cifar10-32x32.zip
python fid.py ref --data=datasets/cifar10-32x32.zip --dest=fid-refs/cifar10-32x32.npz
```

**ImageNet:** Download the [ImageNet Object Localization Challenge](https://www.kaggle.com/competitions/imagenet-object-localization-challenge/data) and convert to ZIP archive at 64x64 resolution:

```.bash
python dataset_tool.py --source=downloads/imagenet/ILSVRC/Data/CLS-LOC/train \
    --dest=datasets/imagenet-64x64.zip --resolution=64x64 --transform=center-crop
python fid.py ref --data=datasets/imagenet-64x64.zip --dest=fid-refs/imagenet-64x64.npz
```

Please see the [EDM](https://github.com/NVlabs/edm) repo for how to construct the FID statistics for CIFAR-10 and ImageNet.



## Generate & Evaluations

- CIFAR-10

  - Step 0: Download the CIFAR-10 checkpoint from [EDM](https://github.com/NVlabs/edm) repo or [PFGM++](https://github.com/Newbeeer/pfgmpp) repo.

  - Step 1: Fill in the Restart configuration in `try_params.txt`, in the format of 

    $$\{``\textrm{idx}(t_{\textrm{min}, i})":[K_i, t_{\textrm{max}, i}, N_{\textrm{Restart}, i}]\}_{i=1}^l, N_{\textrm{main}}$$

    such as `{"14": [10, 0.2964, 3]} 18 `.

  - Step 2: run `python3 hyperparam_cifar.py ` to automate generation and FID evaluation.

- ImageNet $64\times 64$

  - Step 0: Download the ImageNet $64\times 64$ checkpoint from [EDM](https://github.com/NVlabs/edm) repo

  - Step 1: Fill in the Restart configuration in `try_params.txt`, $$\{``\textrm{idx}(t_{\textrm{min}, i})":[K_i, t_{\textrm{max}, i}, N_{\textrm{Restart}, i}]\}_{i=1}^l, N_{\textrm{main}}, \gamma$$

    such as `{"4": [1, 40.7864, 4], "10": [1, 1.92, 4], "11": [5, 1.088, 4], "12": [5, 0.5853, 4], "14": [6, 0.2964, 6]} 36 0.05`

  - Step 2: run `python3 hyperparam.py ` to automate generation and FID evaluation.

