CategoricalGibbsMetropolis is slower than the older ElemwiseCategoricalStep

[Anjum48]

Looking at Austin Rochford's Dirichlet process mixture model which used the older ElemwiseCategoricalStep, it looks 20,000 iterations took just over 2 minutes.

Comparing to the same example in the docs which uses the newer CategoricalGibbsMetropolis, the same number of iterations takes over 14 minutes.

Does anyone know why there is such an increase in run time and if there's a way to make it faster? I've just updated my code (with the new step method) with uses the same model but on much larger data sets, and what used to take a few days to run now looks like it might span weeks :(

[twiecki]

That's unfortunate. It's probably related to the Python loop where each element is proposed and accepted individually, rather than all at once like the previous Elemwise did (did we remove that sampler? Why not still use that?). Maybe there's a way to theanoize that loop or we bring back the older one as an alternative.

[Anjum48]

It looks like ElemwiseCategorical seems to do the trick however there is a warning saying it's depreciated

[ozankabak]

I agree with Thomas, it probably has to do with the Python loop. BTW, ElemwiseCategorical does not support various operations (e.g. indexing), hence the message. But this is a good indication that it should be kept around until CategoricalGibbsMetropolis's performance catches up.

[twiecki]

Actually it's probably not just the python loop, but the fact that logp gets evaluated N times for a vector of length N, rather than just once. There's no way around that for Gibbs but it does suggest that we should keep ElemwiseCategorical, but maybe rename it to CategoricalMetropolis for consistency.

[rahuldave]

It is indeed super-slower. It makes doing clustering Models nigh impossible. See this screenshot for time differences on the old faithful waiting times 2 component gaussian mixture model..

[twiecki]

I wonder if a theano.scan could speed the loop up because it could allow theano to cache the rest of the graph.

@rahuldave Can you just use ElemwiseCategorical? The best solution would be to use a marginalize the cluster assignments: http://pymc-devs.github.io/pymc3/notebooks/marginalized_gaussian_mixture_model.html

[rahuldave]

Yes thats what I did. But perhaps the competences ought to currently be changed? Marginalizing, both explicitly and implicitly using the pymc API is part of tomorrows lab :-)

[twiecki]

Good point about changing the default.

…

On Apr 14, 2017 3:36 AM, "Rahul Dave" ***@***.***> wrote: Yes thats what I did. But perhaps the competences ought to currently be changed? Marginalizing, both explicitly and implicitly using the pymc API is part of tomorrows lab :-) — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#1563 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AApJmGVc2X8lc5-pDQv5VK_6AxPZERbwks5rvs20gaJpZM4K_Cp8> .

[twiecki]

See #2037.