# Human-Guided Exploration (HuGE)

This repository provides an implementation of the Human Guided Exploration (HuGE) algorithm, as proposed in *Breadcrumbs to the Goal: Goal-Conditioned Exploration from Human-in-the-Loop Feedback*
The manuscript is available on [arXiv](TODO)

If you use this codebase, please cite 

Bibtex source is provided at the bottom of the Readme.

Note: in the code sometimes we refer to HuGE as hugrl


## Setup

Clone the lexa-benchmark in the dependencies folder:

```cd dependencies
cd dependencies
git clone git@github.com:orybkin/lexa-benchmark.git
cd ..
```

Conda

```
conda env create -f environment.yml
conda develop dependencies
conda develop dependencies/lexa_benchmark
conda develop dependencies/ravens
pip install -e dependencies/Locobot-env
pip install -e dependencies/Painting-locobot
```

## Example script

```
python launch_main.py --env_name pointmass_rooms --method hugrl
```

#### Methods available:
- hugrl
- human
- oracle
- gcsl

#### Benchmarks available:
- pointmass_rooms
- kitchenSeq
- pusher_hard
- complex_maze
- ravens_pick_and_place
- bandu


If you have, and would like to use, a GPU, you will need to additionally install a GPU-compiled version of PyTorch. To do so, simply run

```
pip uninstall torch && pip install torch==1.1.0
```

## Development Notes

The directory structure currently looks like this:

- huge (Contains all code)
    - envs (Contains all environment files and wrappers)
    - algo (Contains all GCSL code)
        - huge.py (implements high-level algorithm logic, e.g. data collection, policy update, evaluate, save data)
        - buffer.py (The replay buffer used to *relabel* and *sample* (s,g,a,h) tuples
        - networks.py (Implements neural network policies.)
        - variants.py (Contains relevant hyperparameters for GCSL)
- . (Contains all the launcher files)
- doodad (We require this old version of doodad)
- dependencies (Contains other libraries like rlkit, rlutil, room_world, multiworld, etc., and our libraries for the painting (Painting-Locobot) and pick and place (Locobot-env) real world environments)

Please file an issue if you have trouble running this code.


## Bibtex
#### TODO
