Include conditional autoregression (CAR) as multivariate distribution

[ckrapu]

I have been using the implementation of the CAR distribution provided in @junpenglao's notebook and have found that it works pretty well in practice both as a likelihood and as a latent variable prior. I have used it with >10k locations on a GPU with some good speedups for both ADVI and NUTS. It would be nice if it were included out-of-the-box with PyMC3; I have done some instructional classes on spatial models and it's clunky to have to refer students to the notebook for the implementation.

If there is interest, I would like to submit a PR that includes the CAR as a multivariate distribution in similar form to the implementation from the notebook.

[junpenglao]

That would be awesome! Please go ahead and I am happy to review it ;-)

[ckrapu]

Implementing the .random method for the proposed CAR distribution is challenging in the sense that there are two options (that I am aware of) that have major disadvantages of either poor scaling behavior or including substantial additional code.

1. To sample x such that it has the appropriate CAR covariance matrix S we can simulate x by first generating a random variate z and applying the transformation x = Lz where L is the Cholesky square root of the covariance matrix S. Since this incurs a n^3 computation cost in the dimension of S, this would scale poorly. It does not appear to be possible to use the sparse log determinant likelihood formula to also help.

2. We sample from the Gibbs distribution for x with a Gibbs sampler. This would require including a bespoke sampling algorithm (though probably not very many lines of code) in the codebase which is probably also pretty unappealing.

I'd like anyone's opinion on which road to go down, if any. I am also open to any suggestions for alternative ideas for implementing the random method.

[junpenglao]

Hmm neither sound very satisfying, I suggest leave out the random method for now.