Improve autodata docstring detection

[pakal]

Currently, if a variable (eg. a data dict MY_VARIABLE) is imported from a submodule to a parent module, and that an "autodata" pointing to the parent module's MY_VARIABLE is added to documentation, the initial MY_VARIABLE's docstrings (eg. "#: stuffs" is lost. It doesn't seem possible to add this docstring near the "from submodule import MY_VARIABLE).

Autodata "annotation" can be used to workaround this, but in this case the repr() of the variable (that I wanted to be in docs) doesn't appear anymore.

It would be nice for sphinx to follow imports and find the docstring of imported variables, or that it uses a force-injected "doc" one would put in such variables (I tried, it didn't work), or that there is a way to force a display of the repr() of a variable.

[pakal]

It might be related to the troubles of #857 btw

[tk0miya]

Reasonable. But it is hard to get the original location of the variable.

[pakal]

Yes I think the easiest would be to rely on :annotation:, but offer a parameter to "repr()" the variable anyway (useful to autodocument default settings and the likes), wouldn't it ?

[tk0miya]

I think most difficult thing is obtaining comment for the variable. How to rendering is not problem in this case. What do you mean "easy"?

[pakal]

I mean the ":annotation:" parameter in RST sources seems good enough, imho, to provide documentation for the data when the original docstring can't be found directly in python source.
Actually, all we need is a way to force a repr() even when an ":annotation:" is present, and that'd be good enough for everyone I think.

[tk0miya]

It's not good idea. The :annotation: parameter is only used to give right-hand side annotations from out side. For example, it is used for type of return values of functions (ex. -> int), assigned value (ex. = 1) and so on. It means :annotation: parameter is a part of signature, not a document body (in other word, content). In addition, it was introduced for internal use. So it is still undocumented option.

If you can write autodata directive manually, you can give contents for the directive manually.

.. autodoc:: modname.CONST

   document for CONST

You don't need to use any workaround for this.

[pakal]

Alright I was mislead by the sphinx docs mentioning "autodata" : http://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#directive-autoattribute

Maybe the doc should be updated to reflect what you just wrote? But I'm still a bit confused about the real usecases for :annotation:, so I wouldn't know how to turn it exactly. Anyway putting a document body lets the repr() visible in sphinx docs, so it's perfect for my current usecase.

[tk0miya]

Ah, sorry. My last comment was wrong.

In addition, it was introduced for internal use. So it is still undocumented option.

:annotation: option has been described in our document as you pointed. So it is public option. I confused with other options.

And I pushed a PR #6530 to fix the document. I hope this helps you. I guess no :annotation: option will resolve your case. Could you check it please?

[pakal]

Looks good to me :)

[tk0miya]

Thank you for reviewing!

[kconnour]

I'd just add one thing that I haven't seen discussed (here or elsewhere): if importing a variable that's type hinted, the type's docstring is what Sphinx displays (not the repr()). Example from package pkg:

constants.py:

foo: float = 12.34
""" My docstring for foo """

__init__.py:

from .constants import *

Then in my documentation if I try to use
.. autodata:: pkg.foo
it displays: "Convert a string or number to a floating point number, if possible."

but if I use
.. autodata:: pkg.constants.foo
it displays what I'd expect: "My docstring for foo "

I presume this is due to the difficulty finding where the variable is actually located. In any case I figured it was closely enough related to the original issue to mention here.

[vepain]

I agree with @kconnour: using __init__.py import to hide too deep submodules paths creates a docstring issue.

Even if you explicitly import your constants in __init__.py:

# pkg/__init__.py

from pkg.constants import foo

And even if you use alternative docstring comments:

#: My docstring for foo
foo: float = 12.34

Is there a way to extract correctly the docstring, or tell to sphinx where exactly is the constant (with hiding the real path)?

Thank you for your help.

EDIT:
One idea would be to provide an actual link to the constants, so that autodata can extract the correct docstring; and tell autodata (with an option, such as :modpath:) where the constant would be imported from.

In the previous example, this would give:

.. autodata:: pkg.constants::foo
    :modpath: pkg.foo

However, this is quite ugly.

[smarie]

In the previous example, this would give:

.. autodata:: pkg.constants::foo
:modpath: pkg.foo

However, this is quite ugly.

Also it would not work if this is done through autosummary and not autodata, see #12020