Skip to content

Commit 78dd670

Browse files
ConorMacBrideConorMacBride
authored
Merge pull request #174 from ConorMacBride/docs
Add initial Sphinx docs
2 parents 13425bf + 4cb4a09 commit 78dd670

20 files changed

+861
-0
lines changed

Diff for: ‎.github/workflows/docs.yml

+58
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,58 @@
1+
name: Docs
2+
3+
on:
4+
push:
5+
branches:
6+
- main
7+
tags:
8+
- v[0-9]+.[0-9]+.[0-9]+
9+
pull_request:
10+
11+
permissions:
12+
contents: write
13+
14+
concurrency:
15+
group: ${{ github.workflow}}-${{ github.head_ref }}
16+
cancel-in-progress: true
17+
18+
jobs:
19+
build:
20+
runs-on: ubuntu-latest
21+
steps:
22+
- uses: actions/checkout@v3
23+
- uses: actions/setup-python@v4
24+
with:
25+
python-version: '3.9'
26+
- run: python -m pip install pip --upgrade
27+
- run: python -m pip install -e ".[test,docs]"
28+
- run: |
29+
cd docs
30+
make html
31+
- uses: actions/upload-artifact@v3
32+
with:
33+
name: docs-html
34+
path: |
35+
docs/_build/html
36+
retention-days: 5
37+
38+
deploy:
39+
if: ${{ github.event_name != 'pull_request' }}
40+
needs: [build]
41+
environment:
42+
name: github-pages
43+
runs-on: ubuntu-latest
44+
steps:
45+
- name: Download doc build
46+
uses: actions/download-artifact@v3
47+
with:
48+
name: docs-html
49+
path: ./docs/_build/html
50+
- name: Upload to GitHub Pages
51+
uses: peaceiris/actions-gh-pages@v3
52+
with:
53+
github_token: ${{ secrets.GITHUB_TOKEN }}
54+
publish_dir: ./docs/_build/html
55+
exclude_assets: '.buildinfo,_static/jquery-*.js,_static/underscore-*.js'
56+
destination_dir: ./latest
57+
keep_files: false
58+
full_commit_message: Deploy latest to GitHub Pages

Diff for: ‎.gitignore

+3
Original file line numberDiff line numberDiff line change
@@ -9,3 +9,6 @@ __pycache__
99
build
1010
dist
1111
.tmp
12+
.DS_Store
13+
/docs/sample
14+
/docs/_build

Diff for: ‎.pre-commit-config.yaml

+1
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ repos:
7474
hooks:
7575
- id: isort
7676
args: ["--sp", "setup.cfg"]
77+
exclude: ".*(docs/conf.py)$"
7778
- repo: https://github.com/pre-commit/pre-commit-hooks
7879
rev: v4.3.0
7980
hooks:

Diff for: ‎docs/Makefile

+20
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,20 @@
1+
# Minimal makefile for Sphinx documentation
2+
#
3+
4+
# You can set these variables from the command line, and also
5+
# from the environment for the first two.
6+
SPHINXOPTS ?=
7+
SPHINXBUILD ?= sphinx-build
8+
SOURCEDIR = .
9+
BUILDDIR = _build
10+
11+
# Put it first so that "make" without argument is like "make help".
12+
help:
13+
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
14+
15+
.PHONY: help Makefile
16+
17+
# Catch-all target: route all unknown targets to Sphinx using the new
18+
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
19+
%: Makefile
20+
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

Diff for: ‎docs/_templates/mpl_third_party_sidebar.html

+8
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
<nav class="bd-links" id="bd-docs-nav" aria-label="{{ _('Site navigation') }}">
2+
<p class="bd-links__title" role="heading" aria-level="1">
3+
<a href="{{ pathto('index') }}">{{ project }} Documentation</a>
4+
</p>
5+
<div class="bd-toc-item navbar-nav">
6+
{{ generate_toctree_html("sidebar", startdepth=0, maxdepth=1, collapse=True, includehidden=True, titles_only=True) }}
7+
</div>
8+
</nav>

Diff for: ‎docs/conf.py

+74
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,74 @@
1+
# Configuration file for the Sphinx documentation builder.
2+
#
3+
# This file only contains a selection of the most common options. For a full
4+
# list see the documentation:
5+
# https://www.sphinx-doc.org/en/master/usage/configuration.html
6+
7+
import datetime
8+
from packaging.version import Version
9+
10+
from pytest_mpl import __version__
11+
12+
# -- Path setup --------------------------------------------------------------
13+
14+
# If extensions (or modules to document with autodoc) are in another directory,
15+
# add these directories to sys.path here. If the directory is relative to the
16+
# documentation root, use os.path.abspath to make it absolute, like shown here.
17+
#
18+
import os
19+
import sys
20+
sys.path.insert(0, os.path.abspath('.'))
21+
22+
23+
# -- Project information -----------------------------------------------------
24+
25+
project = 'pytest-mpl'
26+
author = 'Thomas Robitaille'
27+
copyright = '{}, {}'.format(datetime.datetime.now().year, author)
28+
29+
release = __version__
30+
pytest_mpl_version = Version(__version__)
31+
is_release = not (pytest_mpl_version.is_prerelease or pytest_mpl_version.is_devrelease)
32+
33+
34+
# -- General configuration ---------------------------------------------------
35+
36+
# Add any Sphinx extension module names here, as strings. They can be
37+
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
38+
# ones.
39+
extensions = [
40+
'sample_summaries',
41+
'sphinx_design',
42+
]
43+
44+
# Add any paths that contain templates here, relative to this directory.
45+
templates_path = ['_templates']
46+
47+
# List of patterns, relative to source directory, that match files and
48+
# directories to ignore when looking for source files.
49+
# This pattern also affects html_static_path and html_extra_path.
50+
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
51+
52+
53+
# -- Options for HTML output -------------------------------------------------
54+
55+
# The theme to use for HTML and HTML Help pages. See the documentation for
56+
# a list of builtin themes.
57+
#
58+
html_theme = "mpl_sphinx_theme"
59+
html_theme_options = {
60+
"navbar_links": "absolute",
61+
"show_prev_next": False,
62+
"logo": {"link": "https://matplotlib.org/stable/",
63+
"image_light": "images/logo2.svg",
64+
"image_dark": "images/logo_dark.svg"},
65+
"collapse_navigation": False,
66+
}
67+
html_sidebars = {
68+
"**": ["mpl_third_party_sidebar.html", "sidebar-nav-bs.html"]
69+
}
70+
71+
# Add any paths that contain custom static files (such as style sheets) here,
72+
# relative to this directory. They are copied after the builtin static files,
73+
# so a file named "default.css" will overwrite the builtin "default.css".
74+
# html_static_path = ['_static']

Diff for: ‎docs/configuration.rst

+187
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,187 @@
1+
.. title:: Configuration
2+
3+
#############
4+
Configuration
5+
#############
6+
7+
Tolerance
8+
^^^^^^^^^
9+
10+
The RMS tolerance for the image comparison (which defaults to 2) can be
11+
specified in the ``mpl_image_compare`` decorator with the ``tolerance``
12+
argument:
13+
14+
.. code:: python
15+
16+
@pytest.mark.mpl_image_compare(tolerance=20)
17+
def test_image():
18+
...
19+
20+
Savefig options
21+
^^^^^^^^^^^^^^^
22+
23+
You can pass keyword arguments to ``savefig`` by using
24+
``savefig_kwargs`` in the ``mpl_image_compare`` decorator:
25+
26+
.. code:: python
27+
28+
@pytest.mark.mpl_image_compare(savefig_kwargs={'dpi':300})
29+
def test_image():
30+
...
31+
32+
Baseline images
33+
^^^^^^^^^^^^^^^
34+
35+
The baseline directory (which defaults to ``baseline`` ) and the
36+
filename of the plot (which defaults to the name of the test with a
37+
``.png`` suffix) can be customized with the ``baseline_dir`` and
38+
``filename`` arguments in the ``mpl_image_compare`` decorator:
39+
40+
.. code:: python
41+
42+
@pytest.mark.mpl_image_compare(baseline_dir='baseline_images',
43+
filename='other_name.png')
44+
def test_image():
45+
...
46+
47+
The baseline directory in the decorator above will be interpreted as
48+
being relative to the test file. Note that the baseline directory can
49+
also be a URL (which should start with ``http://`` or ``https://`` and
50+
end in a slash). If you want to specify mirrors, set ``baseline_dir`` to
51+
a comma-separated list of URLs (real commas in the URL should be encoded
52+
as ``%2C``).
53+
54+
Finally, you can also set a custom baseline directory globally when
55+
running tests by running ``pytest`` with::
56+
57+
pytest --mpl --mpl-baseline-path=baseline_images
58+
59+
This directory will be interpreted as being relative to where pytest
60+
is run. However, if the ``--mpl-baseline-relative`` option is also
61+
included, this directory will be interpreted as being relative to
62+
the current test directory.
63+
In addition, if both this option and the ``baseline_dir``
64+
option in the ``mpl_image_compare`` decorator are used, the one in the
65+
decorator takes precedence.
66+
67+
Results always
68+
^^^^^^^^^^^^^^
69+
70+
By default, result images are only saved for tests that fail.
71+
Passing ``--mpl-results-always`` to pytest will force result images
72+
to be saved for all tests, even for tests that pass.
73+
74+
When in **hybrid mode**, even if a test passes hash comparison,
75+
a comparison to the baseline image will also be carried out,
76+
with the baseline image and diff image (if image comparison fails)
77+
saved for all tests. This secondary comparison will not affect
78+
the success status of the test.
79+
80+
This option is useful for always *comparing* the result images against
81+
the baseline images, while only *assessing* the tests against the
82+
hash library.
83+
If you only update your baseline images after merging a PR, this
84+
option means that the generated summary will always show how the
85+
PR affects the baseline images, with the success status of each
86+
test (based on the hash library) also shown in the generated
87+
summary. This option is applied automatically when generating
88+
a HTML summary.
89+
90+
When the ``--mpl-results-always`` option is active, and some hash
91+
comparison tests are performed, a hash library containing all the
92+
result hashes will also be saved to the root of the results directory.
93+
The filename will be extracted from ``--mpl-generate-hash-library``,
94+
``--mpl-hash-library`` or ``hash_library=`` in that order.
95+
96+
Base style
97+
^^^^^^^^^^
98+
99+
By default, tests will be run using the Matplotlib 'classic' style
100+
(ignoring any locally defined RC parameters). This can be overridden by
101+
using the ``style`` argument:
102+
103+
.. code:: python
104+
105+
@pytest.mark.mpl_image_compare(style='fivethirtyeight')
106+
def test_image():
107+
...
108+
109+
Package version dependencies
110+
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
111+
Different versions of Matplotlib and FreeType may result in slightly
112+
different images. When testing on multiple platforms or as part of a
113+
pipeline, it is important to ensure that the versions of these
114+
packages match the versions used to generate the images used for
115+
comparison. It can be useful to pin versions of Matplotlib and FreeType
116+
so as to avoid automatic updates that fail tests.
117+
118+
Removing text
119+
^^^^^^^^^^^^^
120+
121+
If you are running a test for which you are not interested in comparing
122+
the text labels, you can use the ``remove_text`` argument to the
123+
decorator:
124+
125+
.. code:: python
126+
127+
@pytest.mark.mpl_image_compare(remove_text=True)
128+
def test_image():
129+
...
130+
131+
This will make the test insensitive to changes in e.g. the freetype
132+
library.
133+
134+
Test failure example
135+
--------------------
136+
137+
If the images produced by the tests are correct, then the test will
138+
pass, but if they are not, the test will fail with a message similar to
139+
the following::
140+
141+
E Exception: Error: Image files did not match.
142+
E RMS Value: 142.2287807767823
143+
E Expected:
144+
E /var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmp4h4oxr7y/baseline-coords_overlay_auto_coord_meta.png
145+
E Actual:
146+
E /var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmp4h4oxr7y/coords_overlay_auto_coord_meta.png
147+
E Difference:
148+
E /var/folders/zy/t1l3sx310d3d6p0kyxqzlrnr0000gr/T/tmp4h4oxr7y/coords_overlay_auto_coord_meta-failed-diff.png
149+
E Tolerance:
150+
E 10
151+
152+
The image paths included in the exception are then available for
153+
inspection:
154+
155+
+----------------+----------------+-------------+
156+
| Expected | Actual | Difference |
157+
+================+================+=============+
158+
| |expected| | |actual| | |diff| |
159+
+----------------+----------------+-------------+
160+
161+
In this case, the differences are very clear, while in some cases it may
162+
be necessary to use the difference image, or blink the expected and
163+
actual images, in order to see what changed.
164+
165+
The default tolerance is 2, which is very strict. In some cases, you may
166+
want to relax this to account for differences in fonts across different
167+
systems.
168+
169+
By default, the expected, actual and difference files are written to a
170+
temporary directory with a non-deterministic path. If you want to instead
171+
write them to a specific directory, you can use::
172+
173+
pytest --mpl --mpl-results-path=results
174+
175+
The ``results`` directory will then contain one sub-directory per test, and each
176+
sub-directory will contain the three files mentioned above. If you are using a
177+
continuous integration service, you can then use the option to upload artifacts
178+
to upload these results to somewhere where you can view them. For more
179+
information, see:
180+
181+
* `Uploading artifacts on Travis-CI <https://docs.travis-ci.com/user/uploading-artifacts/>`_
182+
* `Build Artifacts (CircleCI) <https://circleci.com/docs/1.0/build-artifacts/>`_
183+
* `Packaging Artifacts (AppVeyor) <https://www.appveyor.com/docs/packaging-artifacts/>`_
184+
185+
.. |expected| image:: images/baseline-coords_overlay_auto_coord_meta.png
186+
.. |actual| image:: images/coords_overlay_auto_coord_meta.png
187+
.. |diff| image:: images/coords_overlay_auto_coord_meta-failed-diff.png

Diff for: ‎docs/images/baseline-coords_overlay_auto_coord_meta.png

26.5 KB
Loading

Diff for: ‎docs/images/coords_overlay_auto_coord_meta-failed-diff.png

9.13 KB
Loading

Diff for: ‎docs/images/coords_overlay_auto_coord_meta.png

23.4 KB
Loading

Diff for: ‎docs/images/html_all.png

60.1 KB
Loading

Diff for: ‎docs/images/html_filter.png

56.6 KB
Loading

Diff for: ‎docs/images/html_result.png

56.3 KB
Loading

0 commit comments

Comments
 (0)