Skip to content

added --no-run option for rustdoc #83857

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 11 commits into from
May 1, 2021
Merged

added --no-run option for rustdoc #83857

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
db6a916
added --no-run option
ABouttefeux Apr 4, 2021
0f3efe2
Update src/librustdoc/lib.rs from suggestion
ABouttefeux Apr 5, 2021
a6bf81b
suggestion from review move no run option in run_test call
ABouttefeux Apr 5, 2021
72f534a
fix failing build
ABouttefeux Apr 5, 2021
b08b484
Merge branch 'master' into master
ABouttefeux Apr 5, 2021
08c7f97
correction based on review
ABouttefeux Apr 6, 2021
3ad3597
change based on reviews
ABouttefeux Apr 7, 2021
4770446
add test
ABouttefeux Apr 7, 2021
202659a
fix failing tidy check
ABouttefeux Apr 7, 2021
3273d2f
error when --no-run is present without --test
ABouttefeux Apr 30, 2021
03c710b
Apply suggestions from code review
ABouttefeux May 1, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
38 changes: 38 additions & 0 deletions src/test/rustdoc-ui/no-run-flag.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
// test the behavior of the --no-run flag

// check-pass
// compile-flags:-Z unstable-options --test --no-run --test-args=--test-threads=1
// normalize-stdout-test: "src/test/rustdoc-ui" -> "$$DIR"
// normalize-stdout-test "finished in \d+\.\d+s" -> "finished in $$TIME"

/// ```
/// let a = true;
/// ```
/// ```should_panic
/// panic!()
/// ```
/// ```ignore (incomplete-code)
/// fn foo() {
/// ```
/// ```no_run
/// loop {
/// println!("Hello, world");
/// }
/// ```
/// fails to compile
/// ```compile_fail
/// let x = 5;
/// x += 2; // shouldn't compile!
/// ```
/// Ok the test does not run
/// ```
/// panic!()
/// ```
/// Ok the test does not run
/// ```should_panic
/// loop {
/// println!("Hello, world");
/// panic!()
/// }
/// ```
pub fn f() {}
12 changes: 12 additions & 0 deletions src/test/rustdoc-ui/no-run-flag.stdout
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@

running 7 tests
test $DIR/no-run-flag.rs - f (line 11) ... ok
test $DIR/no-run-flag.rs - f (line 14) ... ignored
test $DIR/no-run-flag.rs - f (line 17) ... ok
test $DIR/no-run-flag.rs - f (line 23) ... ok
test $DIR/no-run-flag.rs - f (line 28) ... ok
test $DIR/no-run-flag.rs - f (line 32) ... ok
test $DIR/no-run-flag.rs - f (line 8) ... ok
Comment on lines +3 to +9
Copy link
Contributor

@pickfire pickfire Apr 8, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it didn't run why is it "ok"? IIRC that is the same as test passing right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so adding the --no-run flag change the behavior of the test like it was annotated as ```no_run. Maybe this is unwanted and something else should be displayed.

Copy link
Contributor

@pickfire pickfire Apr 8, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ping @CraftSpider @jyn514 should we have a different text? "ok" may be confusing since it didn't run. Also note that we may also want a different color or the same color as "ignored".

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm. I don't think 'ignored' is quite right, but 'ok' is also a bit confusing. If we can make it something new, maybe something like 'built' or 'checked'? If not, I think 'ok' is more accurate than 'ignored'.
Color-wise, I'm fine with it being ok colored, as it is a requested change to behavior, not implicit, so it's requested to treat it as success. If that makes sense.

Copy link
Contributor Author

@ABouttefeux ABouttefeux Apr 9, 2021 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After investigations I think that in order to do so, I have to modify the lib test. This will also affect all tests and not just doctests. Do you think this is a worthwhile investment and should this be in a different PR ? Anyway I would be interested in implementing that but the scope is a bit bigger than I anticipated.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no_run by itself is useless.
To summarize:

  • if you run rustdoc -Z unstable-options --no-run it just build the documentation, it has the same behavior as just rustdoc
  • rustdoc -Z unstable-options --no-run --test run the doctest set. It behave as if every test had the no_run meaning they won't run. Meaning that test like
    ```
    panic!()
    ```
    will print as ok like
    ```no_run
    panic!()
    ```
    without the --no-run flag

I agree that is confusing. Maybe a error should be raised if no_run is active and not should_test. However I think that if you run rustdoc -Z unstable-options --no-run your intention was probably to run the test instead of documenting your project.

I assume that in the original issue the final goal would be to be able to generate bin (with --persist-doctests) but without running them.

Or maybe a another possible case would be to unify the behavior of cargo test --no-run (provided that down the line cargo is also modified). For instance cargo test --all test both regular test and doctest but cargo test --all --no-run skips doctest cargo test --doc --no-run give an error.

Now if the explicit goal is to harmonize with regular test the output should just be empty.
Another option would for me to change the test lib such that no_run test display something like test foo ... checked, provided that such change is desirable (to discuss thought the RFC process ?).
Moreover test with compile_fail should also print something different maybe checked is also desirable.

I am new contributing for rust so I am not quite sure of what to do to solve this issue.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree that is confusing. Maybe a error should be raised if no_run is active and not should_test. However I think that if you run rustdoc -Z unstable-options --no-run your intention was probably to run the test instead of documenting your project.

I think we should give a hard error here; if people meant to run the test they can add --test. Could you suggest adding --test in the error message?

See

rust/src/librustdoc/config.rs

Lines 587 to 593 in 202659a

} else if !out_fmt.is_json() && show_coverage {
diag.struct_err(
"html output format isn't supported for the --show-coverage option",
)
.emit();
return Err(1);
}
for an example of how to give an error if the options are inconsistent.

I think all your other changes can go in a follow-up PR, I don't think we should be changing libtest here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok thank you. I have not change libtest yet I will work on that too :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do not change libtest in this pr. I am not a maintainer and don't know who to ask to approve; finding out will delay landing this.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry I was not clear. I will do another PR for libtest


test result: ok. 6 passed; 0 failed; 1 ignored; 0 measured; 0 filtered out; finished in $TIME