This paper presents an open source interval reproduction task for RL agents that is based on psychophysics tasks originally developed in neuroscience. This task is used in the paper to better understand how different RL agents solve timing tasks, for example by examining action trajectories and unit activations. This work establishes an open tool and a framework that could be used by future studies to understand computations related to timing in various RL agents. The reviewers all agreed that this paper provides a worthwhile contribution to both the machine learning and neuroscience communities. They had some initial concerns related to generality and scalar variability. But, the reviewers were happy with the author responses on the issues raised and all agreed that this was a good paper that passed the bar for acceptance at NeurIPS.