Document how to make pytest-mpl optional

[cdeil]

One of the issues we struggled with for astroplan was how to make the image comparison tests optional, i.e. have them skipped if matplotlib or nose as optional dependencies aren't installed.

@astrofrog – What do you think about the solution @eteq came up with?
(see https://github.com/astropy/astroplan/blob/master/astroplan/conftest.py)

def pytest_configure(config):
    if hasattr(astropy_pytest_plugins, 'pytest_configure'):
        # sure ought to be true right now, but always possible it will change in
        # future versions of astropy
        astropy_pytest_plugins.pytest_configure(config)

    #make sure astroplan warnings always appear so we can test when they show up
    warnings.simplefilter('always', category=AstroplanWarning)

    try:
        import matplotlib
        import nose  # needed for the matplotlib testing tools
        HAS_MATPLOTLIB_AND_NOSE = True
    except ImportError:
        HAS_MATPLOTLIB_AND_NOSE = False

    if HAS_MATPLOTLIB_AND_NOSE and config.pluginmanager.hasplugin('mpl'):
            config.option.mpl = True
            config.option.mpl_baseline_path = 'astroplan/plots/tests/baseline_images'

@astrofrog – I'd like to set up image comparison tests for other packages in the near future and have them optional. Could you please document the recommended setup?

I think it would also be good to have a short list of "packages using pytest-mpl" ... there's nothing better than working examples to get started.

[astrofrog]

@cdeil - sorry for the long delay. This seems fine, but I'm hesitant to have a single 'recommended' way to deal with this. The pytest-mpl package has required dependencies of matplotlib (and nose for matplotlib < 2.0) so to some extend if pytest-mpl is installed but matplotlib is not, this is not a supported use case?

[ConorMacBride]

Maybe it would be good to include some ways of doing this in a documentation website for a v1 release. The next major/minor version of pytest will show a warning if a test function returns anything other than None, so if people are to simply not install pytest-mpl they'll get a warning (due to the returned figures not being intercepted by pytest-mpl). We could document this at least.

Alternatively, if the following is integrated into setup.cfg, the figure tests will be skipped by default:

[tool:pytest]
addopts = -m "not mpl_image_compare"
markers =
    mpl_image_compare: compares returned figure to baseline

Then, -m mpl_image_compare can be added to the pytest command (inside CI or tox configs) to make only the figure tests run. (Or -m "_ or not _" can be added to make all the tests run.)