multivariate mixture random method is broken

[junpenglao]

https://discourse.pymc.io/t/shape-mismatch-error-in-sample-ppc/2270

The problem is the weight random is incorrect:
https://github.com/pymc-devs/pymc3/blob/6d07591962a6c135640a3c31903eba66b34e71d8/pymc3/distributions/mixture.py#L160-L164

[junpenglao]

OK, the problem is more serious...
Currently, we are generating categorical latent index from weight and component random from comp_dist, if the weight is a vector, we index the component random: comp_random[..., weight_random], but it is incorrect as we should generate new sample for comp_random for each slice...

dist = pm.NormalMixture.dist(w=np.array([.5, .25, .25]), 
                             mu=np.array([-1., 1., 0]), 
                             sd=1, 
                             shape=10)
dist.random()

array([ 0.43626031,  1.59448573,  0.43626031, -0.45223364, -0.45223364,
       -0.45223364,  0.43626031, -0.45223364,  0.43626031,  1.59448573])

As you can see above, we have identical value when the latent index is the same, but it shouldnt be the case...

[lucianopaz]

@junpenglao, I was trying to fix this problem today. I managed to get the broadcasting working for sampling from the posterior predictive. To make sure that the broadcasting was working well, I also wanted to try to sample from the prior predictive but got stumped by a separate problem which maybe is not important for this issue. The LKJCholeskyCov distribution has no random method. Does it make sense to try to implement it from LKJCorr.random? I was trying to do that and got into so much more shape problems

[junpenglao]

Yes it make sense if you are up for the challenge ;-) you need to generate from LKJCorr and the diagnoal dist and than combine them together.