Docs now build successfully out of the box on macOS CPython.

[tmontes]

Exploring possibilities of collaboration, a following up on this conversation at #470, I found out that I could not build the documentation on my laptop, running macOS with vanilla installations of CPython 3.6 and 3.7 from www.python.org.

The failure is in the docs/jsonschema_role.py Sphix extension, in the fetch_or_load function, which uses urllib.request.urlopen with an HTTPS URL. Unfortunately, such CPython installs fail to validate server-side TLS certificates.

The motive stems from the fact that, as stated in the installer's README, a private OpenSSL copy is included in the distribution which does not use the system certificate store for validation; instead, the suggested approach is doing a system-wide installation of the certifi third-party package and then creating a symlink from a file in that package to a file in the standard library (argh!).

Since I explicitly avoid installing any system-wide Python packages -- and, thus, completing the above mentioned install -- I often need to find ways to make things work. I believe many others experience similar failures due to a) actively understanding the issue and not installing system-wide things (like me), or b) not being aware of the issue and not reading the installer README where a possible solution is documented and suggested (many/most people, I'd venture).

However urllib.request.urlopen can be explicitly given a cafile argument which, again explicitly in the context of jsonschema, can use certifi's CA list. This PR adds certifi as a requirement to build docs and adjusts the urlopen call such that is uses certifi's CA list.

With this in place, the docs can be build on any platform, regardless of the (unfortunately somewhat sad) state of the underlying Python's TLS certificate validation mechanism / dependencies.

Feedback is welcome, of course.
Thanks.

[Julian]

Cool! Thanks, so two comments --

Firstly, purely on the matter of "supporting" CPython as a development platform, I don't use it -- I'm totally happy to "support" it (i.e. guarantee that this and related stuff "works" when developing jsonschema), but would love for that to be tested, to ensure it doesn't break, especially given that it's not an environment I'm likely to test under locally. Long story short, I think for this, it'd be nice to add building the docs under CPython to the tox.ini if that's a thing that we want to make sure always works.

I suspect that making the tox.ini build the docs on multiple interpreters will clutter up this PR a bit, so to me probably perfectly reasonable to do it "right after" this one, but yeah lemme know if that makes sense to you. I don't for example necessarily think we need to support building the docs (or development in general) on multiple versions of Python (beyond PyPy2.latest, PyPy3.latest, CPython2.latest, Cpython 3.latest).

Overall... Obviously I get queazy around security things :)

Everything you said makes total sense, but I wonder whether it's even safer to just decide to use requests there instead? We're anyways adding a doc-dep.

[tmontes]

Firstly, purely on the matter of "supporting" CPython as a development platform, I don't use it -- I'm totally happy to "support" it (i.e. guarantee that this and related stuff "works" when developing jsonschema), but would love for that to be tested, to ensure it doesn't break, especially given that it's not an environment I'm likely to test under locally.

Out of curiosity, if you don't develop on CPython, which interpreter(s) do you use, pypy/pypy3? Or do you mean you don't develop on CPython on macOS?

Long story short, I think for this, it'd be nice to add building the docs under CPython to the tox.ini if that's a thing that we want to make sure always works.

That makes sense, yes. For now, like I commented yesterday, the environment I prepared is not using tox so I just built the docs manually with cd docs && pip install -r requirements.txt && make html.

I suspect that making the tox.ini build the docs on multiple interpreters will clutter up this PR a bit, so to me probably perfectly reasonable to do it "right after" this one, but yeah lemme know if that makes sense to you. I don't for example necessarily think we need to support building the docs (or development in general) on multiple versions of Python (beyond PyPy2.latest, PyPy3.latest, CPython2.latest, Cpython 3.latest).

Those platforms make sense, yes. Even a 3-only subset would make sense, I'd say, regarding "developing documentation", but maybe that's a stretch...

Overall... Obviously I get queazy around security things :)

Everything you said makes total sense, but I wonder whether it's even safer to just decide to use requests there instead? We're anyways adding a doc-dep.

I gave that a quick thought and said to myself "urlopen is good enough, no need for more dependencies, even if doc-building only". Less is more, I said. :)

[tmontes]

All AppVeyor and Travis CI jobs failed... :/

• Most due to the fact that test jsonschema.tests.test_jsonschema_test_suite.TestDraft7.test_time_validation_of_time_strings_a_valid_time_string is failing...
• The pypy one on Travis here failed for a different motive.

Does this make any sense to you?

[Julian]

I just reverted the time test, it's related to that Format Checker issue I mentioned in that comment. On Jan 11, 2019 12:11, "Tiago Montes" <[email protected]> wrote: All AppVeyor and Travis CI jobs failed... :/ - Most due to the fact that test jsonschema.tests.test_jsonschema_test_suite.TestDraft7.test_time_validation_of_time_strings_a_valid_time_string is failing... - The *pypy* one on Travis here <https://travis-ci.org/Julian/jsonschema/jobs/478426085> failed for a different motive. Does this make any sense to you? — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#516 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAUIXlpPCMCADvoqtJiY4k6rcPoewhFwks5vCMXOgaJpZM4Z7x4A> .

[Julian]

Uh sorry and as for

The pypy one on Travis here failed for a different motive.

That is pyga/ebb-lint#13 which I now have admin rights to so have merged the PR, but in the process of getting access to PyPI to do a release to fix it. I hope that'll happen in the next few days, but if not we can pin intervaltree<3 to fix it temporarily.

[codecov]

Codecov Report

Merging #516 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #516   +/-   ##
=======================================
  Coverage   95.68%   95.68%           
=======================================
  Files          19       19           
  Lines        2364     2364           
  Branches      308      308           
=======================================
  Hits         2262     2262           
  Misses         89       89           
  Partials       13       13

[Julian]

Out of curiosity, if you don't develop on CPython, which interpreter(s) do you use, pypy/pypy3? Or do you mean you don't develop on CPython on macOS?

pypy yeah. I haven't particularly used CPython for anything other than "check what CPython does on this code" in quite a number of years now, so while of course it's supported for end-users (who haven't learned yet to use PyPy :) I don't generally deal with it on a day to day basis.

And this is green now with the two fixes, so think that's a good to merge then, though yeah lemme know if you can add the tox.ini things.