Write rustdoc

[brson]

We probably want our own system to extract documentation from source code. Graydon has mentioned in the past an interest in using syntax extension for this, but a javadoc analogue seems fine.

The two general-purpose systems I've looked into, doxygen and naturaldocs, both have significant downsides. Doxygen requires a filter to translate the source code to something C++-like so I suspect would never give very satisfying results. Naturaldocs isn't widely used and, by default, doesn't understand enough about the structure of the code to establish the correct hierarchies. It might be made to work more nicely with some effort.

Right now we're using naturaldocs just to get something working, but I think we should just roll our own - that seems how most languages do it.

[elly]

I hacked up an extraordinarily rough prototype: http://www.leptoquark.net/~elly/rustdoc.rs (output when run on itself: http://www.leptoquark.net/~elly/rustdoc.html). Do people have opinions on this docstring-like way of documenting functions? Would attributes be cooler?

[brson]

I think that commandeering the language syntax like that is kind of perverted, it will have some impact on every compiler pass, and it doesn't seem very extensible. I don't know that attributes are a good choice either because they are fairly heavy, but I would prefer them to creating a record like that. I imagine graydon would have an opinion on this subject.

[elly]

Does @graydon work on github?

[jckarter]

I find documentation embedded in source code tends to make both the code and the docs hard to read. What if a module's documentation were kept in a separate file associated with the module file? That approach works well in Factor. It also lets you use an off-the-shelf markup language instead of needing to invent a javadoc-like convention for embedding markup.

[graydon]

Interesting but I think confusing when done with that syntax.

I'd be more inclined towards doing it via an attribute, for example:

#[doc = "frob the thing t"]
frob_it(t: thing) {
...
}

I am somewhat fond of this because we have most of the machinery already built to encode doc-attributes into crates, meaning that (unless someone strips them or, say, passes a flag to omit them) you can ask rustc to print a documented signature from the crate binary directly.

I was always a little worried that by not having "header files" we would be losing the casual habit of shipping comments-as-docs along with an API. I think it'd be a nice touch.

Of course, there's not much difference between that and /** ... */, aside from the latter being a syntax rustc doesn't know yet and would have to learn to consume, attach to items properly and reproduce. Whereas it already knows how to do that with attrs.

[brson]

I like the idea that attribute-based docs are automatically embedded in crates. Here are some possible syntaxes that work in the current system.

#[doc = "Frob the thing"]

#[doc = "Frob the thing

Upon calling the thing is passed through the frobnicator, which frobs,
defrobs, and changes it's fur color to the currently active default color.

This kills the thing.
"]

#[doc(

desc = "Frob the thing",

args(t = "The thing that must be frobbed",
     f = "The frobnicator"),

ret = "The thing, utterly frobbed"

)]

The doc interpreter could be quite permissive in what it understands about doc attributes, allowing for very simple or very structured docs.

[brson]

Attributes can also be put inside the items they describe, which some people prefer.

[brson]

There could also be a doc pass which converts javadoc to doc attributes.

[graydon]

Yeah. I'm a bit torn. I hate to make decisions like this based on "what's easy to implement", but the fact is that comment-preservation and manipulation -- much less "associating a comment with an item" -- is much less solid in the compiler than the treatment of attributes, so in the near-term it feels like attributes are the thing that will "work" here without serious front-end surgery. And attributes already have, as brson points out, a lightweight structured syntax in case you want your docs to have additional fiddly bits (per-parameter docs, warnings, notes, see-also links, deprecated flags, etc.)

On the other hand, I recognize that javadoc /** ... */ style doc-comments have become somewhat de facto in many languages in this family.

How about we start with an attribute syntax and if people complain we can add a secondary mode that considers /** ... / as a special compiler-supported syntax for #[doc=...]? So long as the AST node marks whether the attr came from /* ... / or not, it should be possible to pretty-print in much the same place we permit attributes. Just requires recognizing /* at the start of a comment block.

[graydon]

Elly submitted a preliminary rustdoc tool with f2715f3, so closing this. File subsequent bugs against rustdoc (will add a tag).