You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat: Refactor api-markdown-documenter to be written in terms of ASTs and functional transformations (#14201)
## BREAKING CHANGE
This is a complete overhaul of the library, greatly changing its API
surface. The general consumption patterns remain mostly the same, but
the largely operate on our new AST types.
## New Architecture
The old architecture was based on (and greatly re-used) the base
API-Documenter library. This PR moves mostly away from this. Rather than
operating on a flow that resembled something like:
```
ApiItem (w/ DocNode) -> DocNode -> stateful render
```
We now transform in input ApiItem tree (generated by API-Extractor) into
an [AST](https://en.wikipedia.org/wiki/Abstract_syntax_tree)-based
Documentation Domain. From there, we provide a more-functional rendering
library for rendering that domain as Markdown. So something more like:
```
ApiItem -> Documentation Domain -> functional render
```
### Benefits
- Functional renderer is much easier to understand, easy to unit test
(see the `RenderFoo` test modules), and made it easy to clean up a
number of rendering inconsistencies hidden by the API-Documenter-based
`MarkdownEmitter` renderer.
- Intermediary Documentation Domain is easy to understand, `AST`
compatible, and requires no auxiliary state to process (old DocNode
domain required a tsdoc context to be able to walk, and also had to
reach back into the ApiItem domain to resolve links, etc.).
- Uses existing [unist](https://github.com/syntax-tree/unist) library
for AST representation (with domain-specific type-safety).
## Notes for reviewers
The actual code diff is probably not all that useful. Some of the
configuration types will look roughly the same, but otherwise the
majority of the code has been re-written.
I would recommend referring to the README's architectural overview /
changes to the prescribed workflow.
Additionally, you can view the diffs in our rendering snapshot tests to
confirm that behavioral changes are relatively small, and in my opinion,
are all improvements.
---------
Co-authored-by: Justin Myhres <[email protected]>
Co-authored-by: Justin Myhres <[email protected]>
Copy file name to clipboardExpand all lines: tools/api-markdown-documenter/README.md
+102-16
Original file line number
Diff line number
Diff line change
@@ -2,6 +2,10 @@
2
2
3
3
Contains a programmatic API for generating [Markdown][] documentation from an API report generated by [API-Extractor][].
4
4
5
+
> Note: this library is specifically configured to target [GitHub flavored Markdown][].
6
+
>
7
+
> Compatibility with other Markdown flavors is not guaranteed, though you can always write your own renderer implementation against our [Documentation Domain](#documentation-domain)!
8
+
5
9
It is similar to [API-Documenter][] and is heavily based upon it and uses it under the hood, but is designed to be more extensible and can be used programmatically.
6
10
7
11
**Note**: this library does not currently offer a Command Line Interface (CLI).
@@ -27,7 +31,7 @@ npm i @fluid-tools/api-markdown-documenter -D
27
31
28
32
## Usage
29
33
30
-
### Quickstart
34
+
### Quick Start
31
35
32
36
This library is intended to be highly customizable.
33
37
That said, it is also intended to be easy to use out of the box.
@@ -38,20 +42,20 @@ Are you already generating `.api.json` report files as a part of your build?
38
42
If yes, create a file called `api-markdown-documenter.js` and paste the following code:
The above script can be invoked as an `npm` script by adding the following to your `package.json`'s `scripts` property:
@@ -67,17 +71,22 @@ For more advanced usage options, see the following sections.
67
71
68
72
This package contains 2 primary, programmatic entry-points for generating documentation:
69
73
70
-
#### renderDocuments
74
+
#### transformApiModel
71
75
72
-
The `renderDocuments` function accepts an [ApiModel][] representing the package(s) of a repository, and generates a sequence of "Document" objects representing the resulting documentation based on the other provided configuration options.
76
+
The `transformApiModel` function accepts an [ApiModel][] representing the package(s) of a repository, and generates a sequence of "Document"[Abstract Syntax Tree][] objects representing the resulting documentation based on the other provided configuration options.
73
77
These objects include information about the page item, its documentation contents, and the intended output file path the document file should be rendered to, based on provided policy options.
74
78
79
+
- These trees are backed by [unist][]'s AST model.
80
+
75
81
The input `ApiModel` here will generally be the output of [API-Extractor][].
76
82
77
-
#### renderFiles
83
+
See [Documentation Domain](#documentation-domain) below for more details on the output format.
84
+
85
+
#### renderApiModelAsMarkdown
86
+
87
+
The `renderApiModelAsMarkdown` function operates like [transformApiModel](#createdocuments), but writes the resulting documents to disk as files based on the provided configuration options.
78
88
79
-
The `renderFiles` function operates like [renderDocuments](#renderdocuments), but writes the resulting documents to disk as files based on the provided configuration options.
80
-
This function also accepts a `MarkdownEmitter` object that does the conversion from `DocNode` trees to a Markdown stream that will ultimately be rendered to file.
89
+
This function accepts overrides for all of its default `Markdown`-rendering behaviors, so feel free to customize as you see fit!
81
90
82
91
### Loading the API Model
83
92
@@ -87,10 +96,10 @@ To generate an API model from `.api.json` files generated by `API-Extractor`, se
87
96
88
97
### Emitting Markdown Content
89
98
90
-
If you are using the [renderDocuments](#renderdocuments) option described above, one option for emitting Markdown string content is to use the `emitMarkdown` function.
99
+
If you are using the [transformApiModel](#createdocuments) option described above, one option for emitting Markdown string content is to use the `emitMarkdown` function.
91
100
It accepts a `MarkdownDocument` object as input, and outputs a string representing the final Markdown content.
92
101
93
-
Note: you can also accomplish this by just using [renderFiles](#renderfiles) if you are using default configuration / emitter options.
102
+
Note: you can also accomplish this by just using [renderApiModelAsMarkdown](#renderfiles) if you are using default configuration / emitter options.
94
103
95
104
### Configuration Options
96
105
@@ -107,25 +116,98 @@ The vast majority of these options have default values that have been crafted to
107
116
-`uriRoot`: This is the root URI under which the generated documentation will be published.
108
117
It will be used when generating links for API content docs.
109
118
119
+
## Architectural Overview
120
+
121
+
As noted above, this library is intended to be consumed programmatically.
122
+
While we may at some point add some command line interfaces for common paths, the primary goal of this library's architecture is to be flexible and extensible.
123
+
To that end, we have broken its logic into a few isolable steps, each offering its own extensibility offerings.
124
+
125
+
End to end, this library can be viewed as a psuedo-functional transformation pipeline mapping from an [APIModel][] generated by `API-Extractor` to one or more `Markdown`-formatted document files.
126
+
But this is broken into the following internal sequences:
127
+
128
+
```mermaid
129
+
graph LR
130
+
A[ApiModel]
131
+
B[Documentation AST]
132
+
C[Markdown]
133
+
134
+
A -- transformApiModel --> B
135
+
B -- renderDocumentAsMarkdown --> C
136
+
A -.- renderApiModelAsMarkdown -.-> C
137
+
```
138
+
139
+
For more details on the interior `Documentation AST` ([Abstract Syntax Tree][]) domain, see [Documentation Domain](#documentation-domain) below.
140
+
141
+
### API-Extractor
142
+
143
+
The input to our system is the [ApiModel][] generated by [API-Extractor][].
144
+
145
+
This library offers the `loadModel` function as an option for loading your model from the generated `.api.json` metadata files `API-Extractor` generates be default.
146
+
147
+
To transform this input to our [Documentation Domain][], you may call the `transformApiModel` function.
148
+
This function walks the input `ApiModel` tree, transforming each `ApiItem` under it according to its configured series of transformation policies.
149
+
These policies are entirely configurable, though the system offers defaults out of the box.
150
+
151
+
### Documentation Domain
152
+
153
+
This library defines its own "Documentation Domain" using [Abstract Syntax Tree][] (AST) syntax backed by [unist][].
154
+
This is used as an intermediate domain between the `API-Extractor`-based input, and the `Markdown` rendering output.
155
+
This domain was crafted to support [TSDoc][]'s capabilities, and to represent something resembling lowest common denominator between `Markdown` and `HTML` syntax.
156
+
157
+
As this domain is implemented as an `AST`, it is highly customizable.
158
+
If you are interested in creating your own intermediate domain concepts, feel free to implement them.
159
+
So long as you provide a corresponding [renderer policy](#markdown-renderer), the system will gladly accept them!
160
+
161
+
### Markdown Renderer
162
+
163
+
The final component of this library's transformation pipeline is its `Markdown` renderer.
164
+
165
+
> Note: by default, this renderer is configured to generate [GitHub flavored Markdown][].
166
+
>
167
+
> Compatibility with other Markdown flavors is not guaranteed.
168
+
169
+
As with the other logic in this library, the renderer is highly configurable.
170
+
It will accept any `Documentation Domain` tree as input, and transform each node according to its configured render policies.
171
+
172
+
If you would like to add rendering support for a custom `Documentation Domain` node type, simply provide a renderer policy associated with that node's `type` value.
173
+
174
+
If you would like to change any or all of this library's default rendering policies, you may simply override the default policies for the desired `type`s.
175
+
110
176
## Upcoming Work
111
177
178
+
- Simplify input configurations
179
+
- Separate configs for each pipeline segment
180
+
- Clean up terminology ("policies" in particular is too vague - we should remove this)
181
+
- Add extensibility options for `DocNode` transformations
182
+
- If a consumer has a custom tsdoc config associated with their API-Extractor setup, this will be needed.
183
+
184
+
### Known Bugs
185
+
186
+
- Example titles are not respected. See TSDoc [@example spec](https://tsdoc.org/pages/tags/example/) - text on the same line as the tag should be treated as the example title.
187
+
- Types that extend or implement types with generic parameters result in signatures rendered with missing closing `>`s.
188
+
112
189
### Documentation Improvements
113
190
114
191
- Intro sandbox (api report)
192
+
- Extensibility examples (maybe use the AlertNode concept, once we move it out of the library and into the FluidFramework website build)
115
193
116
-
### Styling improvements
194
+
### Styling Improvements
117
195
118
-
- Remove leading blank line in documents
119
-
- Excessive blank lines in Signature sections
120
196
- Fix links to the same file (only need heading component, not file path)
121
197
- This will require plumbing down a context document item, so we can easily determine if the document to which the link is being generated is the same as the document being linked to.
198
+
- Config options for parsing TSDoc block comment contents as Markdown (and don't escape the contents)?
199
+
- Add support for Table Cell alignment
200
+
201
+
### Performance Improvements
202
+
203
+
- Rather than repeatedly walking up a given `ApiItem`'s hierarchy when evaluating paths, links, etc., we could pass down transformation context object containing a running record of the item's hierarchy as we walk the tree.
122
204
123
205
## Longer-term work
124
206
125
-
- Replace DocNode output / MarkdownEmitter with Markdown AST trees and simple interface for rendering those trees to a stream
126
207
- Support placing documents _within_ their own hierarchy (support for the "index" model used by systems like DocFX)
127
208
- Pre-canned policies (flat, index, adjacency)
128
-
- Handle multiple package entrypoints
209
+
- Handle multiple package entry-points
210
+
- Add separate HTML transformation path, for consumers that want to go straight to HTML
0 commit comments