# Asymptotic Properties for Bayesian Neural Network in Besov Space

## Author

* Kyeongwon Lee, lkw1718@snu.ac.kr 
* Jaeyong Lee, leejyc@gmail.com

## Structure

```
.
|-- README.md 
|-- docs           # documents
|-- figs           # fitted model
|-- model          # fitted model
|-- results.ipynb  # ipynb to plot figs
`-- src            # source code for experiments
```

## Experiments

* We estimate each model with NUTS algorithm using python pyro package and PyTorch package (Paszke et al., 2019).
* We implement Bayes By Backprop algorithm based on [JavierAntoran/Bayesian-Neural-Network](https://github.com/JavierAntoran/Bayesian-Neural-Networks) repository. 

### Run

```bash 
python src/example.py --function ["cantor" | "sobolev" | "blocks" | "heavisine"] --n [n] --W [W] --L [L] --prior [prior] --seed [seed] --algorithm ["mcmc" | "BBP"] --epochs [epochs] --lr [learning rate] --tnx [True | False]
```

* `[function]`: true function
* `[W]`: width of the neural network model
* `[L]`: depth of the neural network model
* `[n]`: data sample size
* `[prior]`: prior distribution for the model parameters
* `[seed]`: random seed
* `[algorithm]`: infer model using Markov Chain Monte Carlo (mcmc) or Bayes By Backprop (BBP)
* `[epochs]`: epochs for the initial model (mcmc) or BBP.
* `[learing rate]`: learning rate for the initial model (mcmc) or BBP.
* `[tnx]`: data from truncated normal $N(0, 1) I(0, 1)$ (default: uniform $U(0, 1)$ - False)

for the paper,

* function: "cantor", "sobolev", "blocks" and "heavisine"
* W: 200
* L: 5
* n: 100, 1,000 and 10,000
* prior: "2GMM" and "Gaussian"
* seed: 1
* algorithm: "mcmc" (warmup = 1,000 and sample = 1,000)
* epochs: 10,000 (n=100), 5,000 (n=1,000) and 1,000 (n=10,000)
* learning rate: 2e-3
* tnx: False (uniform distribution)

### Summary

run `results.ipynb`.

