# Clip-OGD Simulations

This repository contains code for reproducing the simulation results in our paper:

> Jessica Dai, Palua Gradu, Christopher Harshaw. "Clip-OGD: An Experimental Design for Adaptive Neyman Allocation in Sequential Experiments". NeurIPS 2023. [arXiv link](https://arxiv.org/abs/2305.17187).

The code provided here is research level code meant only to reproduce simulation results and may not be appropriate for other purposes, e.g. running an actual adaptive experiment.
The code is provided under the Creative Commons Attribution 4.0 International License (CC BY 4.0).

## Files and Directories

Below are a list of the relevant functions:

- `design-funs.jl`: contains functions for sampling from adaptive designs, constructing estimators, and creating summary statistics.
- `create-outcomes.jl`: loads `Groh2016-data.csv` and applies further post-processing, e.g. cleans the data, imputes missing outcomes, and copies units for larger sample size.
- `create-samples.jl`: samples from different designs and stores the relevant result in CSV files.
- `create-plots.jl`: creates and saves plots and tables from the simulation data.

Below are are a list of directories:

- `data/GM/create-data`: contains the imputed data used in our simulations as well as data generated by our simulations.
- `data/GM/create-figures`: contains the figures created from our simulations.
- `data/GM/raw`: contains code for downloading the raw data from Groh \& MacKenzie (2016)

## Reproducing Simulations

Our simulations use the experimental data set collected in the following paper:

> Matthew Groh, David McKenzie.
"Macroinsurance for microenterprises: A randomized experiment in post-revolution Egypt".
Journal of Development Economics.
Volume 118,
2016,
Pages 13-25.
[link](https://www.sciencedirect.com/science/article/abs/pii/S0304387815000942)

If you also use any of the data obtained from this repository, please cite the original authors.
If you use any of the code in this repository, please cite our paper.

There are three steps to reproducing our simulations:

- **Step 0**: (Optional) Download raw experimental data
- **Step 1**: Impute, normalize, and duplicate raw measured outcomes to obtain full simulation outcomes.
- **Step 2**: Generate 50,000 sample estimators from the various experimental designs.
- **Step 3**: Plot figures from simulation data.

### Step 0: Download Raw Data

We provide `data/GM/raw/Groh2016-data.csv`, which is a file containing the raw dataset from Groh and MacKenzie (2016).
However, if you want to reproduce this dataset yourself, you will need to download the original dataset from Groh \& MacKenzie.
That can be done by visiting the [journal website](https://www.sciencedirect.com/science/article/pii/S0304387815000942#s0125) or running the following bash script:

```
cd data/GM/raw
Groh2016-download.sh
```

This creates a directory `JDEFiles` inside of `data/GM/raw`.
This directory contains raw Stata files.
In order to convert the Stata files into a csv file, you will run the script `Groh2016-clean.do` in Stata.
This will generate a file `Groh2016-data.csv`, which you should move to `data/GM/raw` if it is not already there.

### Step 1: Construct Full Simulation Data

To post-process the raw data in the `Groh2016-data.csv` file, run the following Julia script by entering the following command in the terminal:

```
julia create-outcomes.jl
```

This will create the file `po-df.csv` in the directory `data/GM/created-data`, which has the simulated potential outcomes obtained by post-processing the raw data from Groh \& MacKenzie (2016).
However, you will find that we have already placed the file `po-df.csv` in the directory so that you can run Steps 3 and 4 on their own if you would like.

### Step 2: Generate simulation results

In order to generate simulation results, run the following Julia script by entering the following command in the terminal:

```
julia create-samples.jl
```

This will create several files in the `data/GM/created-data` directory which contain summary statistics and estimators from the simulations.
However, you will find that these files are already populated in that directory so you can run Step 4 on its own if you would like.

### Step 4: Plot Figures

Finally, to plot the figures, you need to run the following Julia script by entering the following command in the terminal:

```
julia create-plots.jl
```

This file creates plots and saves them as .pdf files in the `data/GM/created-figures` directory.