rustdoc: Deprecate option --test-args in favor of new insta-stable option --test-arg

[fmease]

Context: #137096 (comment)

[rustbot]

r? @notriddle

rustbot has assigned @notriddle.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[rustbot]

This PR modifies run-make tests.

cc @jieyouxu

[fmease]

@rfcbot merge

[rfcbot]

Team member @fmease has proposed to merge this. The next step is review by the rest of the tagged team members:

• @GuillaumeGomez
• @Manishearth
• @Nemo157
• @aDotInTheVoid
• @camelid
• @fmease
• @jsha
• @notriddle

Concerns:

• insta-stable resolved by rustdoc: Deprecate option --test-args in favor of new insta-stable option --test-arg #139869 (comment)
• insuppressible-warning resolved by rustdoc: Deprecate option --test-args in favor of new insta-stable option --test-arg #139869 (comment)

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

See this document for info about what commands tagged team members can give me.

[fmease]

@rfcbot concern insta-stable

Should the new flag be insta-stable? If we keep the new flag unstable, we can't recommend a replacement for the newly deprecated flag. Should we just soft-deprecate the old flag first?

Potential solution: Only deprecate the flag under --edition=future (-Zunstable-options) and make the new flag unstable (requiring -Zunstable-options).

@rfcbot concern insuppressible-warning

We currently emit an insuppressible deprecation warning (IIRC it's infeasible or even impossible to make this an early buffered lint warning). However, that'll probably lead to a lot of terminal spam for Cargo users? Idk, does Cargo use this flag under the hood or do users usually pass these explicitly via env var RUSTDOCFLAGS / config opt rustdocflags?

[GuillaumeGomez]

Should we tie this change to a new edition?

[fmease]

Should we tie this change to a new edition?

We could. We could deprecate this in all editions (as it's usually done for rustc/rustdoc's flags) and make --test-args a hard-error in the next edition. However, I think that's orthogonal to the actual deprecation.

[GuillaumeGomez]

Then let's go through deprecation and make the removal in the next edition.

[fmease]

However, I think that's orthogonal to the actual deprecation.

Well, on a second thought, we could delay the deprecation by only rejecting --test-args if --edition=future -Zunstable-options is passed and by gating the new --test-arg flag behind --edition=future -Zunstable-options. This would be a solution to the "insta-stable" concern.

Re. --edition=future: #137606

[Manishearth]

Should the new flag be insta-stable? If we keep the new flag unstable, we can't recommend a replacement for the newly deprecated flag. Should we just soft-deprecate the old flag first?

I'm not opposed to this provided we have a nice principled way of doing these flags across the board. So if we spend some time making sure this is what we want, I think we're fine.

[ehuss]

Just FYI, I don't think you'll be able to land this with a warning as-is since it will cause cargo's tests to fail.
(I personally would recommend not warning for at least a few releases.)

[fmease]

Yeah, I noticed that when looking at the CI failure

[fmease]

--test-args is now only soft-deprecated (i.e., the docs and the CLI help output mention it's deprecated but we don't emit any warnings yet). We can re-upgrade to a (hard) deprecation warning once at least Cargo has had the change to migrate to the new flag.

I'll track this in a tracking issue if this FCP goes through successfully.

@rfcbot resolve insuppressible-warning

[fmease]

For the upgrade path, the new flag needs to be (insta-) stable. We could meddle with performing the deprecation in the Future Edition only (--edition=future -Zunstable-options) but that seems unnecessarily roundabout and complex. Also: #139869 (comment).

@rfcbot resolve insta-stable