Fix target for doc test cross compilation #8094

Freax13 · 2020-04-11T09:38:36Z

for target paths Compilation::target only contains the shortened file name, but we need to pass the full path to rustdoc

rust-highfive · 2020-04-11T09:38:38Z

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @ehuss (or someone else) soon.

If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.

Please see the contribution instructions for more information.

ehuss · 2020-04-11T15:17:23Z

Thanks!

It seems like Compilation.host and Compilation.target only exist for doc tests. Perhaps those should just be removed? Then run_doc_tests can just look at the build_config to decide what to do.

Also, can you add a test for this? There are some examples in custom_target.rs. There is documentation for writing tests in CONTRIBUTING and cargo-test-support.

Freax13 · 2020-04-12T11:13:30Z

It seems like Compilation.host and Compilation.target only exist for doc tests. Perhaps those should just be removed? Then run_doc_tests can just look at the build_config to decide what to do.

Compilation is part of the public api though so removing host and target is technically a breaking change

Freax13 · 2020-04-12T11:18:37Z

Also, can you add a test for this? There are some examples in custom_target.rs. There is documentation for writing tests in CONTRIBUTING and cargo-test-support.

there is a similar bug #71002 in rustdoc that needs to get patched first. when that pr is merged I can add tests

ehuss · 2020-04-12T15:31:02Z

Compilation is part of the public api though so removing host and target is technically a breaking change

Cargo's public API isn't stable, so each release is a breaking change.

This reverts commit 060c5e1.

This reverts commit 5a051eb.

Freax13 · 2020-04-15T11:01:53Z

It seems like Compilation.host and Compilation.target only exist for doc tests. Perhaps those should just be removed? Then run_doc_tests can just look at the build_config to decide what to do.

I played around with this and it seems to me that removing those two attribute results in messy/more complex code, so imho it's not worth removing them.

ehuss · 2020-04-15T21:13:11Z

Hm, ok, I see that doctests have the unusual behavior that --target HOST still needs to work.

Perhaps add a comment explaining that, perhaps along the lines of "Use rustc_target() to properly handle JSON spec paths.".

Let me know if you have any issues adding the test.

Freax13 · 2020-04-15T21:21:18Z

Let me know if you have any issues adding the test.

okay so the fix in rustdoc got merged this morning, so I'm guessing it should be in the next nightly version. I'll try adding the tests tomorrow (and the comment regarding rustc_target)

ehuss · 2020-04-16T16:06:58Z

tests/testsuite/cross_compile.rs

+    let target = cross_compile::alternate();
+
+    // This tests the library and runs the doc tests.
+    p.cargo("test -v -Z doctest-xcompile --target")
+        .arg(&target)


I think this should be using a json spec target, since that's what this PR is fixing. I think it would be best for it to go in the custom_target.rs module with the other json spec tests.

yeah that makes sense not sure why I missed that

Freax13 · 2020-04-16T16:46:21Z

the custom targets obviously can't use std, so exiting cleanly is a bit of a challenge. the only way I can think of right now is by using inline assembly and manually calling the system calls to exit. the downside of this approach is that it wouldn't be cross platform

ehuss · 2020-04-16T22:50:16Z

Hm, I'm wondering, are you somehow using json specs to build std? I'm having a hard time imagining an environment where these kinds of tests would even work. (I'm only familiar with using specs with no_std.)

Freax13 · 2020-04-16T22:53:38Z

I'm using it for my os kernel using the bootimage/bootloader crates which procss the compiled binaries into a disk image

ehuss · 2020-04-16T23:04:20Z

Can you say more of how that would work? Without std, I don't think the tests will work (maybe with a custom test runner, but those don't work with rustdoc afaik).

Freax13 · 2020-04-16T23:21:55Z

correct. i'm using custom test frameworks for normal tests. doctests are actually just small programs that all get compiled and executed individually.

usually rustdoc wraps the code in a main method, but if it detects that you wrote your own, it will use that instead. that way you can also use outer attributes like #![no_std] or use extern crate.

so in my case I have some hidden code in the doctests that runs the os initialization and the os uses qemu's exit decice feature to stop itself and signal whether the tests ran successfully. if i remember correctly i think i managed to get it to work at one point by manually fixing all the target path bugs.
i checked some cross compiled binaries earlier and it seems like they were normal elf files (im not sure if thats the default or if it is influenced by some config key i set). those elf files also had an entrypoint, so i was able to open them in a debugger (they pretty much instantly segfault, but thats probably caused by my kernel trying to setup memory and not because they're inherintly broken)

ehuss · 2020-04-17T15:27:20Z

That is an impressive level of dedication to get doctests working! 😄

Considering that getting an actual doctest working will likely be very difficult, perhaps it should just have a test that cargo passes the correct --target flag to rustdoc. So maybe in custom_target_minimal, add something like the following:

    // Ensure that the correct style of flag is passed to --target with doc tests.
    p.cargo("test --doc --target src/../custom-target.json -v -Zdoctest-xcompile")
        .masquerade_as_nightly_cargo()
        .with_stderr_contains("[RUNNING] `rustdoc [..]--target [..]foo/custom-target.json[..]")
        .run();

with the goal just to check that it passes the correct form of --target. How does that sound?

Freax13 · 2020-04-17T15:35:25Z

sounds good to me

ehuss · 2020-04-17T16:02:18Z

Thanks!
@bors r+

bors · 2020-04-17T16:02:19Z

📌 Commit e5d9b03 has been approved by ehuss

bors · 2020-04-17T16:27:53Z

⌛ Testing commit e5d9b03 with merge 6d76d7c...

bors · 2020-04-17T16:45:23Z

☀️ Test successful - checks-azure
Approved by: ehuss
Pushing 6d76d7c to master...

Update cargo, rls ## cargo 17 commits in ebda5065ee8a1e46801380abcbac21a25bc7e755..8751eb3010d4cdb5329b5a6bd2b6d765c95b0dca 2020-04-16 14:28:43 +0000 to 2020-04-21 18:04:35 +0000 - Uplift windows gnu DLL import libraries. (rust-lang/cargo#8141) - Add windows-gnu CI and fix tests (rust-lang/cargo#8139) - Several updates to token/index handling. (rust-lang/cargo#7973) - Add `resolver` opt-in for new feature resolver. (rust-lang/cargo#8129) - Improve error message when running `cargo install .` (rust-lang/cargo#8137) - fix mem replace unused (rust-lang/cargo#8138) - Change `-Cembed-bitcode=no` use to `-Cbitcode-in-rlib=no`. (rust-lang/cargo#8134) - Refactor BuildContext (rust-lang/cargo#8068) - Rename allows_underscores to allows_dashes. (rust-lang/cargo#8135) - Fixed a needless borrow. (rust-lang/cargo#8130) - Add link to changelog in the Cargo book. (rust-lang/cargo#8126) - Fix target for doc test cross compilation (rust-lang/cargo#8094) - Add note about .cargo/config support. (rust-lang/cargo#8125) - Fix pdb uplift when executable has dashes. (rust-lang/cargo#8123) - Hint upgrading for future edition keys (rust-lang/cargo#8122) - Use some fs shorthand functions. (rust-lang/cargo#8124) - Update documentation to mention "config.toml" instead of "config" (rust-lang/cargo#8121) ## rls 1 commits in 2659cbf14bfb0929a16d7ce9b6858d0bb286ede7..7de2a1f299f8744ffe109139f9f1fdf28bfec909 2020-04-14 22:07:24 +0200 to 2020-04-19 22:41:55 +0000 - Update cargo (rust-lang/rls#1663)

This stabilizes the doctest-xcompile feature by unconditionally enabling it. Closes #7040 Closes #12118 ## What is being stabilized? This changes it so that cargo will run doctests when using the `--target` flag for a target that is not the host. Previously, cargo would ignore doctests (and show a note if passing `--verbose`). A wrapper for running the doctest can be specified with the [`target.*.runner`](https://doc.rust-lang.org/cargo/reference/config.html#targettriplerunner) configuration option (which is powered by the `--test-runtool` rustdoc flag). This would typically be something like qemu to run under emulation. It is my understanding that this should work just like running other kinds of tests. Additionally, the [`target.*.linker`](https://doc.rust-lang.org/cargo/reference/config.html#targettriplelinker) config option is honored for using a custom linker. Already stabilized in rustdoc is the ability to [ignore tests per-target](https://doc.rust-lang.org/nightly/rustdoc/write-documentation/documentation-tests.html#ignoring-targets). ## Motivation The lack of doctest cross-compile support has always been simply due to the lack of functionality in rustdoc to support this. Rustdoc gained the ability to cross-compile doctests some time ago, but there were additional flags like the test runner that were not stabilized until just recently. This is intended to ensure that projects have full test coverage even when doing cross-compilation. It can be [surprising](#12118) to some that this was not happening, particularly since cargo is silent about it. ## Risks The cargo team had several conversations about how to roll out this feature. Ultimately we decided to enable it unconditionally with the understanding that most projects will probably want to have their doctests covered, and that any breakage will be a local concern that can be resolved by either fixing the test or ignoring the target. Tests in rust-lang/rust run into this issue, [particularly on android](rust-lang/rust#119147 (comment)), and those will need to be fixed before this reaches beta. This is something I am looking into. Some cross-compiling scenarios may need codegen flags that are not supported. It's not clear how common this will be, or if ignoring will be a solution, or how difficult it would be to update rustdoc and cargo to support these. Additionally, the split between RUSTFLAGS and RUSTDOCFLAGS can be cumbersome. ## Implementation history - rust-lang/rust#60387 -- Support added to rustdoc to support the `--target` flag and runtool and per-target-ignores. - #6892 -- Initial support in cargo. - #7391 -- Added unstable documentation. - #8094 -- Fix target for doc test cross compilation - #8358 -- Fixed regression with `--target=HOST` not working on stable. - #10132 -- Added note about doctests not running (under `--verbose`). - rust-lang/rust#112751 -- Fixed `--test-run-directory` interaction with `--test-runtool`. - rust-lang/rust#137096 -- Stabilization (and rename) of the rustdoc `--test-runtool` and `--test-runtool-arg` CLI args, and drops `--enable-per-target-ignores` unconditionally enabling it. ## Test coverage Cargo tests: - [artifact_dep::cross_doctests_works_with_artifacts](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/artifact_dep.rs#L1248-L1326) -- Checks that doctest has access to the artifact dependencies. - [build_script::duplicate_script_with_extra_env](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/build_script.rs#L5514-L5614) -- Checks that build-script env and cfg values are correctly handled on host versus target when cross running doctests. - [cross_compile::cross_tests](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/cross_compile.rs#L416-L502) -- Basic test that cross-compiled tests work. - [cross_compile::doctest_xcompile_linker](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/cross_compile.rs#L1139-L1182) -- Checks that the linker config argument works. - [custom_target::custom_target_minimal](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/custom_target.rs#L39-L71) -- Checks that `.json` targets work with rustdoc cross tests. - [test::cargo_test_doctest_xcompile_ignores](https://github.com/ehuss/cargo/blob/56c08f84e28d3653ae0c842f331a226738108188/tests/testsuite/test.rs#L4743-L4777) -- Checks the `ignore-*` syntax works. - [test::cargo_test_doctest_xcompile_runner](https://github.com/ehuss/cargo/blob/2603268cda3e32565ac27ee642f2b755fa590bac/tests/testsuite/test.rs#L4783-L4863) -- Checks runner with cross doctests. - [test::cargo_test_doctest_xcompile_no_runner](https://github.com/ehuss/cargo/blob/2603268cda3e32565ac27ee642f2b755fa590bac/tests/testsuite/test.rs#L4869-L4907) -- Checks cross doctests without a runner. Rustdoc tests: - [run-make/doctest-runtool](https://github.com/rust-lang/rust/tree/25cdf1f67463c9365d8d83778c933ec7480e940b/tests/run-make/doctests-runtool) -- Tests behavior of `--test-run-directory` with relative paths of the runner. - [rustdoc/doctest/doctest-runtool](https://github.com/rust-lang/rust/blob/25cdf1f67463c9365d8d83778c933ec7480e940b/tests/rustdoc/doctest/doctest-runtool.rs) -- Tests for `--test-runtool` and `--test-runtool-arg`. ## Future concerns There have been some discussions (rust-lang/testing-devex-team#5) about changing how doctests are driven. My understanding is that stabilizing this should not affect those plans, since if cargo becomes the driver, it will simply need to build things with `--target` and use the appropriate runner. ## Change notes This PR changed tests a little: - artifact_dep::no_cross_doctests_works_with_artifacts was changed now that doctests actually work. - cross_compile::cross_tests was changed to properly check doctests. - cross_compile::no_cross_doctests dropped since it is no longer relevant. - standard_lib::doctest didn't need `-Zdoctest-xcompile` since `-Zbuild-std` no longer uses a target. - test::cargo_test_doctest_xcompile was removed since it is a duplicate of cross_compile::cross_tests I think this should probably wait until the next release cutoff, moving this to 1.89 (will update the PR accordingly if that happens).

fix target

f9f62ff

rust-highfive assigned ehuss Apr 11, 2020

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label Apr 11, 2020

Freax13 added 2 commits April 12, 2020 17:49

remove host & target attributes

5a051eb

fix running doctest for host target

060c5e1

ehuss added S-waiting-on-author Status: The marked PR is awaiting some action (such as code changes) from the PR author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Apr 14, 2020

Freax13 added 2 commits April 15, 2020 12:51

Revert "fix running doctest for host target"

b365b0f

This reverts commit 060c5e1.

Revert "remove host & target attributes"

86d857e

This reverts commit 5a051eb.

Freax13 added 2 commits April 16, 2020 11:35

add clarifying comment

ecffdcc

add tests for doctests with cross compilation

60a61d4

Freax13 force-pushed the fix-target branch from 54028be to 60a61d4 Compare April 16, 2020 09:48

Freax13 added 2 commits April 16, 2020 12:07

make test use nightly

6fc37f5

fix test

4785b64

Freax13 force-pushed the fix-target branch from 176148e to 4785b64 Compare April 16, 2020 11:08

ehuss reviewed Apr 16, 2020

View reviewed changes

Freax13 mentioned this pull request Apr 17, 2020

Add support for doctests rust-osdev/bootimage#52

Merged

add test for doctests with custom target paths

e5d9b03

bors added S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. and removed S-waiting-on-author Status: The marked PR is awaiting some action (such as code changes) from the PR author. labels Apr 17, 2020

bors merged commit 6d76d7c into rust-lang:master Apr 17, 2020

Freax13 deleted the fix-target branch April 17, 2020 16:45

ehuss mentioned this pull request Apr 21, 2020

Update cargo, rls rust-lang/rust#71402

Merged

ehuss added this to the 1.45.0 milestone Feb 6, 2022

ehuss mentioned this pull request Apr 28, 2025

Stabilize doctest-xcompile #15462

Merged

Fix target for doc test cross compilation #8094

Fix target for doc test cross compilation #8094

Uh oh!

Conversation

Freax13 commented Apr 11, 2020

Uh oh!

rust-highfive commented Apr 11, 2020

Uh oh!

ehuss commented Apr 11, 2020

Uh oh!

Freax13 commented Apr 12, 2020

Uh oh!

Freax13 commented Apr 12, 2020

Uh oh!

ehuss commented Apr 12, 2020

Uh oh!

Freax13 commented Apr 15, 2020

Uh oh!

ehuss commented Apr 15, 2020

Uh oh!

Freax13 commented Apr 15, 2020

Uh oh!

ehuss Apr 16, 2020

Choose a reason for hiding this comment

Uh oh!

Freax13 Apr 16, 2020

Choose a reason for hiding this comment

Uh oh!

Freax13 commented Apr 16, 2020

Uh oh!

ehuss commented Apr 16, 2020

Uh oh!

Freax13 commented Apr 16, 2020

Uh oh!

ehuss commented Apr 16, 2020

Uh oh!

Freax13 commented Apr 16, 2020

Uh oh!

ehuss commented Apr 17, 2020

Uh oh!

Freax13 commented Apr 17, 2020

Uh oh!

ehuss commented Apr 17, 2020

Uh oh!

bors commented Apr 17, 2020

Uh oh!

bors commented Apr 17, 2020

Uh oh!

bors commented Apr 17, 2020

Uh oh!

Uh oh!