Python Client Documentation Site Needs Updating

[scottilee]

The Python Kubernetes Client documentation site (https://kubernetes.readthedocs.io/en/latest/) is out-of-date and could use better attention to formatting.

Per our discussion in the Python Client meeting this morning please investigate:

• if someone is working on developing or replacing this site
• if auto-generation of the site is broken or needs to be setup

cc @roycaihw

[micw523]

Looks like we need a markdown engine for Sphinx. tbh if we convert the readme to rst this site should be working again.

@roycaihw , is this website automatically being deployed?

[micw523]

/cc

[roycaihw]

Thanks for pointing this out. The site looks stale with v1.8 APIs, so I don't think it's automatically being deployed.

From the original PR:

• the intention was to have the test catch out-of-date doc, but we aren't getting the signal from travis-ci
• either running test or a script should update the doc and allow the contributor to check-in the up-to-date files; we are missing documentation around how to do this
• I tried running python setup.py build_sphinx locally and it generates some html pages under /doc/build, which matches this config. I'm not sure if python setup.py upload_sphinx would be the next step here to deploy the content

Since the site has been stale for a long time, I'm not sure if anyone is relying on it (I wasn't aware of this site before), but I think it has value in being a nice web page aggregating the READMEs (generated and manually maintained) in this repo

cc @dims Do you have suggestions about how we can improve this site?

[micw523]

I was able to do something like this using our existing README after some exploration.

It's possible to generate the website using Sphinx but likely requires some effort. I'll assign myself for now since it could take a while.
/assign

[oz123]

@micw523 I have experience with rendering markdown in sphinx. Would you like me to work on this?

[micw523]

Sounds good - feel free to reply that you’ve taken the issue.

I think there are a few things we should do:

• the current repo uses Sphinx 1.3 and is way out of date. We should think of maybe moving on to a higher version.
• the main MD and the generated MDs are located all over the place. It could be difficult to use with recommonmark since that pkg looks for root directory only (if I remember it correctly).
• we should also look at the CI and see what the docs test is trying to accomplish.

[scottilee]

Sphinx is at v2.1.2 now: https://pypi.org/project/Sphinx/. We could also discuss moving away from Sphinx as no one seems to know how to use it. https://www.mkdocs.org/ is another Python-based option or we could use a Go-based option such as https://gohugo.io/.

[micw523]

I wouldn't say "no one seems how to use it". It's clear that @oz123 and I have experience using it - although I have relatively little experience using it with md.

Even though md-based solutions might be useful given all our generated docs are now in markdown, sphinx has autoclass and automodule that covers the ground not being covered by the generated docs, such as base/ and util/. My hunch is that it's still worth it to keep using Sphinx.

[oz123]

@scottilee I agree with @micw523. I would also add, that there is almost no matching in quality to sphinx build, and that it is used in almost any large python project out there. Recently the linux kernel migrated it's docs to sphinx and there a other large non python software projects that use sphinx.

[scottilee]

No worries. @oz123 if you decide to work on this can you please also create a README.md that outlines the steps to build and push changes to the documentation site?

[oz123]

@scottilee we can configure readthedocs to automatically build the docs on every push.
Maybe we can also discuss this with @micw523 on slack next Wednesday in the SIG meeting or after it?

@scottilee see:
https://github.com/oz123/python/blob/docs_update/doc/README.md

[scottilee]

@oz123 Thanks for the updates. A couple questions:

1. You said that you could configure readthedocs to automatically build the docs on every push, is that set up with your pull request Docs update #893? If so, can you give an explanation how it works? For instance if I update docs/source/installation.rst is it TravisCI that builds it and pushes it to readthedocs?
2. I see that https://k8s-python.readthedocs.io is up-to-date but https://kubernetes.readthedocs.io is not. Why are we using two URLs, do we no longer have access to the latter URL?
3. Is there anything else we need to do before I close out this issue?

[oz123]

@scottilee @roycaihw

1. Yes and No.
   Yes, it is set up for my k8s-project project ins https://readthedocs.org/.
   This project is set up to build the documentation every time the master branch in gh/oz123/python is update.
   No. It is not happening to gh/kubernetes-client/python because nobody set it, and I have no admin access in readthedocs.org. The person who set it up can invite me or other developers to the project and I (or they) can set it up. I documented the steps in: Docs update #893 (comment)
2. I used k8s-pyhon.rtd.io because I have no access to https://kubernetes.readthedocs.io.
3. That said, the URL for the documentation is misleading. I would expect to find the documentation for k8s in t https://kubernetes.readthedocs.io, but this is the documentation the for python client only. Therefore, I suggest transferring https://kubernetes.readthedocs.io to thee appropriate k8s organization (maybe in the future they would like to build their docs there too), and keep update the documentation for the python client under k8s-python.rtd.io.
   Of course, I will transfer the ownership of k8s-python.rtd.io to the organization and add the current active members to collaborators in rtd.io

[oz123]

Also, there is this issue
swagger-api/swagger-codegen#9630. If someone could collaborate with me on this, maybe we can fix the issue on our side.
I would need help to understand how openapi-gen (swagger-codegen) is tunes and maybe I can fix things true custom functions in sphinx's conf.py

[micw523]

We’re moving away from swagger-codegen in favor of OpenAPI-generator from v11. @oz123 Maybe try forking Roy’s v11a1 branch and test if it works there?

[oz123]

I did. The issue still exists. I will repo the issue with OpenAPI-generator.

[dims]

@oz123 i've added you as maintainer in https://kubernetes.readthedocs.io.

[oz123]

@dims I see you also updated the settings. Thanks!
@scottilee, @micw523 the documentation is now built with every push on the master.
I believe this issue can be closed.

[scottilee]

@oz123 before I close this issue:

1. Now that you have access to https://kubernetes.readthedocs.io will you be pushing changes there? While I understand your point about expecting to see the main Kubernetes documentation at https://kubernetes.readthedocs.io, that will probably never happen since Kubernetes has their own documentation site. Either way we shouldn't have two sites (i.e., https://kubernetes.readthedocs.io and https://k8s-python.readthedocs.io).

2. Can you update https://github.com/kubernetes-client/python/blob/master/doc/README.md to explain that pushes to the doc/ directory automatically get reflected on https://kubernetes.readthedocs.io or https://k8s-python.readthedocs.io (whichever you have it set to). We should also add a note that about who has admin access to the readthedocs site or create an OWNERS file. Lastly can you also add some info about how that directory is setup to push automatically to readthedocs? It's not clear to me how that works (for example where are credentials stored to write to readthedocs?) and I think it would be a good reference to future contributors.

[oz123]

1. I'm not pushing anything. Github has hooks which can notify different services about changes. Currently, this repository has the readthedocs hook configures. I guess it was set up by @dims.
   I can document this.,
   Also, never say never. I've seen some large projects migrate to sphinx (the most notable one is the linux kernel). The documentation system of kubernetes has a lot of potential to be improved. I am contributing there too, and it's my hope that this will be changed.
   https://k8s-python.readthedocs.io is built only on pushes to https://github.com/oz123/python, and this does not get updated all too often. I think this should be kept, because who know. You might see k8s documentation moving there.
   There are already a few unofficial projects documenting k8s there.
2. All pushes trigger a build in kubernetes.readthedocs.io, not just pushes to doc/. That is because sphinx is extracting documentation strings from the whole code base. I am going to add a guide on how to contribute to the documentation. This should answer your questions.

@scottilee @micw523 @roycaihw I suggest you create a rtd.io account, in order to get acceess to the admin area of kubernetes.readthedocs.io.

[micw523]

Hey @oz123, my user name is my github handle.

Having two documentation sites isn't too bad in my opinion. One of them is based on @oz123's personal repo, which is essentially a testing ground for the full documentation sites. This is actually not a bad idea - it saves effort for other contributors to review documentation-related PRs by him.

[oz123]

@micw523 I added you as a maintainer in https://readthedocs.org/dashboard/kubernetes/users/.
I will add the list of the current maintainers to documentation meta in a PR soon.

[scottilee]

@oz123 Can you also give me access to the readthedocs.io page?

Also, let me know if I misunderstood but it it seems like you said pushes to your repo (https://github.com/oz123/python) trigger builds to both readthedocs.io sites but shouldn't we only trigger builds on pushes to this repo (https://github.com/kubernetes-client/python)?

Lastly, any update on adding a guide to updating the documentation? I didn't see any updates to https://github.com/kubernetes-client/python/blob/master/doc/README.md.

Thanks.

[oz123]

@scottilee I'm sorry I can't be helpful regards to your request. I'm only a co maintainer on read the docs.

Pushes to my fork only update my own site.
Pushes to this site update the documents for this site.

Regarding the last question, the readme you link to implicitly assumes one knows how to work with sphinx to create the documentation. I can send a PR with some more information on this regards.