Fix pm.DensityDist bug and incorporate latest upstream changes

.gitignore

@@ -41,3 +41,4 @@ benchmarks/results/
 pytestdebug.log
 .dir-locals.el
 .pycheckers
+

RELEASE-NOTES.md

@@ -1,13 +1,24 @@
 # Release Notes
 
 ## PyMC3 3.9.x (on deck)
-*waiting for contributions*
+
+### Maintenance
+- Fix an error on Windows and Mac where error message from unpickling models did not show up in the notebook, or where sampling froze when a worker process crashed (see [#3991](https://github.com/pymc-devs/pymc3/pull/3991)).
+
+### Documentation
+- Notebook on [multilevel modeling](https://docs.pymc.io/notebooks/multilevel_modeling.html) has been rewritten to showcase ArviZ and xarray usage for inference result analysis (see [#3963](https://github.com/pymc-devs/pymc3/pull/3963))
+
+### New features
+- Introduce optional arguments to `pm.sample`: `mp_ctx` to control how the processes for parallel sampling are started, and `pickle_backend` to specify which library is used to pickle models in parallel sampling when the multiprocessing cnotext is not of type `fork`. (see [#3991](https://github.com/pymc-devs/pymc3/pull/3991))
+- Add sampler stats `process_time_diff`, `perf_counter_diff` and `perf_counter_start`, that record wall and CPU times for each NUTS and HMC sample (see [ #3986](https://github.com/pymc-devs/pymc3/pull/3986)).
+- Extend `keep_size` argument handling for `sample_posterior_predictive` and `fast_sample_posterior_predictive`, to work on arviz InferenceData and xarray Dataset input values. (see [PR #4006](https://github.com/pymc-devs/pymc3/pull/4006) and [Issue #4004](https://github.com/pymc-devs/pymc3/issues/4004).
+- SMC-ABC: add the wasserstein and energy distance functions. Refactor API, the distance, sum_stats and epsilon arguments are now passed `pm.Simulator` instead of `pm.sample_smc`. Add random method to `pm.Simulator`. Add option to save the simulated data. Improves LaTeX representation [#3996](https://github.com/pymc-devs/pymc3/pull/3996)
 
 ## PyMC3 3.9.2 (24 June 2020)
 ### Maintenance
 - Warning added in GP module when `input_dim` is lower than the number of columns in `X` to compute the covariance function (see [#3974](https://github.com/pymc-devs/pymc3/pull/3974)).
 - Pass the `tune` argument from `sample` when using `advi+adapt_diag_grad` (see issue [#3965](https://github.com/pymc-devs/pymc3/issues/3965), fixed by [#3979](https://github.com/pymc-devs/pymc3/pull/3979)).
-- Add simple test case for new coords and dims feature in `pm.Model` (see [#3977](https://github.com/pymc-devs/pymc3/pull/3977)). 
+- Add simple test case for new coords and dims feature in `pm.Model` (see [#3977](https://github.com/pymc-devs/pymc3/pull/3977)).
 - Require ArviZ >= 0.9.0 (see [#3977](https://github.com/pymc-devs/pymc3/pull/3977)).
 
 _NB: The `docs/*` folder is still removed from the tarball due to an upload size limit on PyPi._

docs/source/notebooks/MLDA_benchmarks_tuning.ipynb

@@ -231,9 +231,9 @@
     "            # convert m and c to a tensor vector\n",
     "            theta = tt.as_tensor_variable(parameters)\n",
     "\n",
-    "            # use a DensityDist (use a lamdba function to \"call\" the Op)\n",
+    "            # use a Potential for the likelihood\n",
     "            ll = logl[j]\n",
-    "            pm.DensityDist('likelihood', lambda v: ll(v), observed={'v': theta})\n",
+    "            pm.Potential('likelihood', ll(theta))\n",
     "\n",
     "        coarse_models.append(cmodel)\n",
     "      \n",
@@ -248,8 +248,8 @@
     "        # Convert m and c to a tensor vector\n",
     "        theta = tt.as_tensor_variable(parameters)\n",
     "\n",
-    "        # use a DensityDist (use a lamdba function to \"call\" the Op)\n",
-    "        pm.DensityDist('likelihood', lambda v: logl[-1](v), observed={'v': theta})\n",
+    "        ## use a Potential for the likelihood\n",
+    "        pm.Potential('likelihood', logl[-1](theta))\n",
     "        \n",
     "    return model, coarse_models, true_parameters"
    ]
@@ -2716,7 +2716,7 @@
    "source": [
     "Generally, the optimal subsampling rate depends on the complexity of the fine posterior. The more complex the posterior, the more samples are needed to generate a decent proposal. The reason is that the MLDA sampler is based on the assumption that the coarse proposal samples (i.e. the samples sent from the coarse chain to the fine one) are independent from each other. In order to generate independent samples, it is necessary to run the coarse chain for an adequate number of iterations to get rid of autocorrelation. The more complex the posterior the more iterations are needed and thus a larger subsampling rate.\n",
     "\n",
-    "Note that in cases where you have more than one coarse model/level, MLDA allows you to choose a different subsampling rate for each coarse level (as a list of integers when you instantiate the stepper)."    
+    "Note that in cases where you have more than one coarse model/level, MLDA allows you to choose a different subsampling rate for each coarse level (as a list of integers when you instantiate the stepper)."
    ]
   }
  ],

docs/source/notebooks/MLDA_multilevel_groundwater_flow.ipynb

@@ -352,9 +352,9 @@
     "        # convert m and c to a tensor vector\n",
     "        theta = tt.as_tensor_variable(parameters)\n",
     "\n",
-    "        # use a DensityDist (use a lamdba function to \"call\" the Op)\n",
+    "        # use a Potential for the likelihood\n",
     "        ll = logl[j]\n",
-    "        pm.DensityDist('likelihood', lambda v: ll(v), observed={'v': theta})\n",
+    "        pm.Potential('likelihood', ll(theta))\n",
     "\n",
     "    coarse_models.append(model)\n"
    ]
@@ -599,8 +599,8 @@
     "    # Convert m and c to a tensor vector\n",
     "    theta = tt.as_tensor_variable(parameters)\n",
     "\n",
-    "    # use a DensityDist (use a lamdba function to \"call\" the Op)\n",
-    "    pm.DensityDist('likelihood', lambda v: logl[-1](v), observed={'v': theta})\n",
+    "    # use a Potential for the likelihood\n",
+    "    pm.Potential('likelihood', logl[-1](theta))\n",
     "\n",
     "    # Initialise an MLDA step method object, passing the subsampling rate and\n",
     "    # coarse models list\n",

docs/source/notebooks/MLDA_multilevel_groundwater_flow_blocked.ipynb

@@ -352,9 +352,9 @@
     "        # convert m and c to a tensor vector\n",
     "        theta = tt.as_tensor_variable(parameters)\n",
     "\n",
-    "        # use a DensityDist (use a lamdba function to \"call\" the Op)\n",
+    "        # use a Potential for the likelihood\n",
     "        ll = logl[j]\n",
-    "        pm.DensityDist('likelihood', lambda v: ll(v), observed={'v': theta})\n",
+    "        pm.Potential('likelihood', ll(theta))\n",
     "\n",
     "    coarse_models.append(model)\n"
    ]
@@ -557,8 +557,8 @@
     "    # Convert m and c to a tensor vector\n",
     "    theta = tt.as_tensor_variable(parameters)\n",
     "\n",
-    "    # use a DensityDist (use a lamdba function to \"call\" the Op)\n",
-    "    pm.DensityDist('likelihood', lambda v: logl[-1](v), observed={'v': theta})\n",
+    "    # use a Potential for the likelihood\n",
+    "    pm.Potential('likelihood', logl[-1](theta))\n",
     "\n",
     "    # Initialise an MLDA step method object, passing the subsampling rate and\n",
     "    # coarse models list\n",