# Top Two Algorithms Revisited

The general structure of the code (and some functions) is taken from [this library](https://bitbucket.org/wmkoolen/tidnabbil/src/master/). References to the papers of the authors of this library are already included in the paper.

## Setting up the environment

**Download Anaconda**
```
wget https://repo.continuum.io/archive/Anaconda3-2021.05-Linux-x86_64.sh
sha256sum Anaconda3-2021.05-Linux-x86_64.sh
bash Anaconda3-2021.05-Linux-x86_64.sh
```

**Create environment**
```
conda create --name julia-env
conda activate julia-env
conda install -c conda-forge julia
```

**Download packages**
```
julia
using Pkg;
Pkg.add(["JLD2", "Printf", "JSON", "Dates", "IterTools", "Distributed", "Optim",
         "Random", "LinearAlgebra", "Distributions", "Combinatorics", "CPUTime",
         "StatsPlots", "ArgParse", "Statistics", "StatsBase", "Plots", "Polynomials",
         "Pickle", "ColorSchemes"]);
```

## Experiments

After installing Julia, to run the experiments presented in the paper, you can either use the custom commands (defined below) to only perform a given experiment or you can directly run the `script.sh`. Some experiments are computationally costly, hence we recommend to use more cores than solely four (`-p4`), or to reduce the number of runs (`--Nruns 100`).

```
cd path_to_folder/code
cd npbai
mkdir experiments
```

### All experiments at once

To run our script with four cores:
```
chmod +x script.sh
bash script.sh 4
```
Note that the script doesn't include the plotting functions. Therefore, additional runs of the corresponding plotting functionalities have to be made afterwards (see below). As it is, you might need to be careful in the order in which you call the visualization functions since the plots are created for all the folders in the `experiments` folder, which don't have `.png` inside them.

### One at a time

#### Simple Bernoulli instances

```
cd path_to_folder/code/npbai
julia -O3 -p4 npbai_experiments.jl --expe "all_GK16" --inst "ber_triv" --K 5 --seed 42 --Nruns 5000
julia -O3 -p4 npbai_experiments.jl --expe "all_GK16" --inst "ber_easy" --K 5 --seed 42 --Nruns 5000
julia -O3 -p4 npbai_experiments.jl --expe "all_GK16" --inst "ber_eq3rd" --K 4 --seed 42 --Nruns 5000
julia -O3 -p4 npbai_experiments.jl --expe "all" --inst "ber_triv" --K 5 --seed 42 --Nruns 1000
julia -O3 -p4 npbai_experiments.jl --expe "all" --inst "ber_easy" --K 5 --seed 42 --Nruns 1000
julia -O3 -p4 npbai_experiments.jl --expe "all" --inst "ber_eq3rd" --K 4 --seed 42 --Nruns 1000
julia npbai_viz_experiments.jl
```

#### Random Bernoulli instances

```
cd path_to_folder/code/npbai
julia -O3 -p4 npbai_randoms.jl --expe "random_GK16" --inst "distinct" --K 8 --seed 42 --Nruns 5000
julia -O3 -p4 npbai_randoms.jl --expe "random_GK16" --inst "distinct" --K 10 --seed 42 --Nruns 5000
julia -O3 -p4 npbai_randoms.jl --expe "random" --inst "distinct" --K 10 --seed 42 --Nruns 1000
julia npbai_viz_randoms.jl
```

#### DSSAT instances

```
cd path_to_folder/code/npbai
julia -O3 -p4 npbai_experiments.jl --expe "dsat_exp_GK16" --inst "dsat_5" --seed 42 --Nruns 500
julia -O3 -p4 npbai_experiments.jl --expe "dsat_exp_GK16" --inst "dsat_6" --seed 42 --Nruns 250
julia -O3 -p4 npbai_experiments.jl --expe "dsat_exp" --inst "dsat_4" --seed 42 --Nruns 100
julia npbai_viz_experiments.jl --plot_hist
```

#### Failure Equality Bernoulli instances

```
cd path_to_folder/code/npbai
julia -O3 -p4 npbai_experiments.jl --expe "fail_equ_GK16" --inst "fail_eq2nd_easy" --K 3 --seed 42 --Nruns 5000
julia -O3 -p4 npbai_experiments.jl --expe "fail_equ_GK16" --inst "fail_eq2nd_okay" --K 3 --seed 42 --Nruns 5000
julia -O3 -p4 npbai_experiments.jl --expe "fail_equ_GK16" --inst "fail_eq2nd_hard" --K 3 --seed 42 --Nruns 5000
julia -O3 -p4 npbai_experiments.jl --expe "fail_equ" --inst "fail_eq2nd_hard" --K 3 --seed 42 --Nruns 1000
julia npbai_viz_experiments.jl --print_table
```
