Executive summary
-----------------

This distribution contains code to reproduce the results in:

    Slice sampling covariance hyperparameters of latent Gaussian models.
    Iain Murray and Ryan Prescott Adams.

    Advances in Neural Information Processing Systems (NIPS) 23, 2010
    http://books.nips.cc/
    http://homepages.inf.ed.ac.uk/imurray2/pub/10hypers/

Much of the code here is detail required to run the experiments. The main
methods in this paper are implemented in update_theta_*.m and contain
documentation.


Some details
------------

This distribution contains the code that was used to produce the reported
results. We did not wish to change anything in case a "small change"
accidentally broke the code, or misrepresented what we did. As a result
there are some known imperfections, which we outline here. We also give
some details for which the paper had no room.

The function names don't match the names used in the paper. Here is the
correspondence:

    ---------------------------------------------
    Paper's name       Matlab function
    ---------------------------------------------
    fixed              update_theta_simple.m
    prior-white        update_theta_aux_chol.m
    surr-              update_theta_aux_surr.m
    post-              update_theta_aux_fixed.m
    ---------------------------------------------

When it gets down to the linear algebra, the surr- and post- methods are
very similar. There are various ways of computing (factorizations of) the
approximate-posterior covariance $R_\theta$, which vary slightly in
numerical stability and computation time. Unfortunately our implementations
use slightly different methods for the surr- and post- methods, for no real
reason. This is evident in the small timing differences on the Gaussian
problem, which we would expect to be more similar.

The surr-site, surr-taylor, post-site and post-taylor methods result from
calling the above functions with different auxiliary noise covariances. The
code refers to auxiliary noise covariance $S_\theta$ through diagonal
standard deviations "aux". (Generalizing to arbitrary covariances would be
trivial, but take more computer time.)

    ------------------------------------------------
    Paper's name   Likelihood   Matlab aux function
    ------------------------------------------------
    -site          Logistic     logistic_aux.m
                   Gaussian     sqrt(diag(K))
                   Poisson      poiss_aux.m
    -taylor        Logistic     N/A
                   Gaussian     sqrt(diag(K))
                   Poisson      poiss_aux_fixed.m
    ------------------------------------------------

Inner-loop approximations: the logistic_aux uses a numerical approximation
to the precise moment matching. However the MCMC method is still valid. The
Laplace approximation to the Poisson site posterior involves the Lambert-W
function. The lambertw function in Matlab is *really* slow, so we started
to approximate further. We later provided our own lambertw routine, making
the straight-up Laplace approximation feasible. It turns out the code we
ran does something slightly different for zero counts. It would probably be
better to use the Laplace approximation throughout, but we have left in
what the reported results actually used. Future users should try the
commented out "Straight-up Laplace approx" in poiss_aux.m.

CARE: There are some rescaling issues to get right, which we dealt with in
the run_ and setup_ functions for each application. For example, the
logistic_aux function assumes there is a logistic likelihood with gain 1
and the GP has variable signal (marginal) variance. In the simulations we
use a fixed signal variance and variable gain, because this is equivalent
but computationally more efficient. However, we had to be careful to
convert between the representations correctly. Some rescaling issues arise
with the Cox process: does the GP function represent the log-intensity of
the process, or the log-rate for the Poisson distribution in each bin?
Sometimes one view is easier/faster and sometimes the other.

A final warning is that add_call_counter.m is inefficient and shouldn't be
used if you care about speed.


Distribution
------------

Code written by the authors of the paper is available under the standard
MIT license. However, various other files by other authors are included.
For full attributions and license terms see COPYING.


Dependencies:
-------------

For convenience we have included the datasets that we used and a copy of
(an old version of) the GPML toolbox with authors and distribution terms
given in its README file.

As it says in the paper, we ran this code on Matlab 7.8. Somewhat earlier
versions will work, but 7.x is probably required. It should all work in
recent versions of Octave too, although this hasn't been tested for a
while. Some of the code may currently depend on Un*x (Linux/Mac) in a
trivial way.

effective_size_rcoda.m depends on R and R-CODA being installed and in your
path. We don't have a Windows machine with R to get the system calls
working there.

