Code

For convenience, the supplemental material contains the code submitted for review. However, for more recent code and additional documentation, please have a look at our GitHub: https://github.com/ir-lab/LanguagePolicies

The source code of our model is given the Code directory of our supplemental material. The following tree outlines the major source code files.

  • main.py $\rightarrow$ Used to train our model with the default hyperparameters.
  • model_src
    • attention.py $\rightarrow$ Our attention model
    • basismodel.py $\rightarrow$ The implementation of our basis model used to generate the trajectories
    • feedbackcontroller.py $\rightarrow$ This is our overall control model used as a cell in a recurrent architecture
    • glove.py $\rightarrow$ Embedding lookup into GloVe
    • model.py $\rightarrow$ Main file of the model, combining all components
  • ros2
    • src
      • policy_translation $\rightarrow$ Defining a ROS 2 node to generate the ROS service files used by our model to communicate between CoppeliaSim and the network service.
  • service.py $\rightarrow$ ROS 2 node that loads a pre-trained model and provides a service that can be used by CoppeliaSim to actuate the robot.
  • utils
    • collect_data.py $\rightarrow$ Code used to collect our training data from CoppeliaSim
    • data_processing_rss.py $\rightarrow$ Data processing: Raw data to TFRecords
    • graphs.py $\rightarrow$ Plotting functions for TensorBoard
    • network.py $\rightarrow$ Wrapper class for our neural network. This class handles the training and validation loop.
    • tf_util.py $\rightarrow$ Utility file to configure TensorFlow depending on different hardware requirements
    • voice.py $\rightarrow$ Templating engine to automatically generate task instructions during the collection of training data
  • val_model_vrep.py $\rightarrow$ CoppeliaSim interface to validate a pre-trained model (together with service.py as control backend)
  • viz_val_vrep.py $\rightarrow$ Takes the val_results.json file generated by val_model_vrep.py to generate a single line of Latex for our main results table.