Automate rustc(1) man page generation using help2man script
#150066
Conversation
Generate the rustc.1 man page from `rustc --help -v` using help2man to ensure the documentation stays in sync with the compiler’s actual CLI behavior. The generation logic lives in src/doc/man and writes rustc.1 to the same directory, independent of the current working directory.
- Introduce `src/doc/man/generate-rustc-man.sh`, a robust script that generates the `rustc(1)` man page entirely from the current built binary’s `--help -v` output using help2man. - Add `README.md` in the same directory to document the rationale, instructions, and prerequisites for this process. - Ensures the man page always reflects the actual compiler behavior, avoids manual edits or duplication, and is easy to update as CLI options evolve.
rustbot
added
the
S-waiting-on-review
|
rustbot has assigned @GuillaumeGomez. Use |
|
Not too sure who's supposed to review this. r? compiler |
rustbot
added
the
T-compiler
|
I’m a bit unsure how to route and review this change, since it was redirected to a compiler team. As far as I understand it introduces new documentation-generation tooling and touches on how the rustc(1) man page is produced, and it’s not clear to me which team should be reviewing this or how it fits into our existing documentation processes Could you clarify:
That would help figure out how (and by whom) this should be discussed further |
|
I noticed two different issues(#149875 and #98977) that mentioned the I apologize for not linking the issues initially, that may have made it more clear. |
|
@Kivooeo i’m willing to review this if you can’t find a reviewer. i don’t have r+ permissions anymore though. |
Yeah, that's sounds really nice, thanks for taking time on it also if you know, because I don't if we have to cooperate this with our release or infra teams to integrate this into our release process, like generate man before/with new release or only after editing cli flags, you might know this better
Also not a problem Do "r=me,jyn514" after approval, I'd only ask to be impartial as you can (for example README here is obviously written by ai, I'd prefer to see something more human if you share this. If it's important to clarify, I'm not writing this to criticize the author, but rather as something that could be improved, nothing personal) r? jyn514 |
|
|
|
@bors delegate=jyn514 |
There was a problem hiding this comment.
The setup in this PR is that people have to manually run the script in order to regenerate the man page, which is better than the current situation but not by much. I would like to see the following things before this gets merged:
- A test that the man page stays up to date
- A way to
--blessthat test with bootstrap
I believe you have not yet run this script since I don't see changes to the man page itself. Please do that and commit the diff so I can see what output it generates.
I think one way you could implement the test is as a run-make test that depends on help2man? I'm not sure if there's an existing way for run-make tests to depend on a command to be installed ... cc @jieyouxu, do you have opinions on this should work?
rustbot
added
S-waiting-on-author
|
Reminder, once the PR becomes ready for a review, use |
Rewrite generate-rustc-man.sh to produce a complete, up-to-date man page by combining help2man output with dynamically generated sections for codegen options and static information
|
Ok, I've updated the script to dynamically generate the CODEGEN OPTIONS section from For the test + bless functionality: I'm happy to either wait for input on how run-make tests should work or I can start implementing a test now if you'd prefer. Let me know which approach you'd like! |
|
The job Click to see the possible cause of the failure (guessed by this bot) |
| gsub(/=val$/, "", option) | ||
| printf ".TP\n\\fB-C %s\\fR=\\fIval\\fR\n%s\n", option, desc | ||
| }' | ||
| } |
There was a problem hiding this comment.
is there a way to get help2man to pass a custom flag here rather than open-coding something in bash? if we're doing it for codegen flags i'd rather do it consistently in rust throughout so we don't need an external dependency.
There was a problem hiding this comment.
If I understand what you are saying, are you suggesting to add a flag to the rustc --help command(eg. --format-man) that will output all of the necessary information for help2man without needing to parse it from external sources like the rust -C help?
If so, I think that makes sense and I'm happy to do it, but correct me if I'm wrong.
There was a problem hiding this comment.
no, i'm asking:
a) does help2man support running rustc -C help instead of --help when it generates this output? if so, you could delete this bash and replace it with a call to help2man.
b) otherwise, if it doesn't support that, then i would like to avoid using help2man altogether and rewrite this bash logic into rust. we'd get rid of the bash script all together and put the rust code in a run-make test. we wouldn't modify the compiler.
either way, i don't want to have a bunch of awk no one knows how to maintain.
does that makes sense?
There was a problem hiding this comment.
Yes, I understand what you are saying. I looked into it and it doesn't look like help2man supports two separate help options. You can specify a custom help option(which I use to specify verbose help output) but you can't have two and concatenate without using some separate logic to string them together, and it sounds like that's what you are trying to avoid, which makes sense.
I'm happy to rewrite this in Rust and use some custom logic to parse the help outputs rather than help2man if that's what you think the best next step.
There was a problem hiding this comment.
You can specify a custom help option(which I use to specify verbose help output)
I'm wondering now if it makes sense to change rustc to accept both rustc --help -C help at once ... but that will be a big hassle since it has to go through an MCP. I think open-coding it in rust is not so bad, that will give us more flexibility in case help2man doesn't recognize some syntax we start using in the future.
yeah ok, let's do the rewrite into rust.
There was a problem hiding this comment.
I've opened rust-lang/compiler-team#954 since I think that's a good change regardless of what we do here, I don't mind if you wait until we get a resolution there to work more on this.
There was a problem hiding this comment.
I think I will wait until we have a decision there to start working more on the rust implementation.
There was a problem hiding this comment.
👍 the MCP got approved. so i think we can simplify this quite a bit:
- extend rustc so that
—help -C helpshows both help outputs - change help2man to use both those flags in a single invocation
- add a UI test for the rustc change and a run-make test that the man page is up-to-date.
There was a problem hiding this comment.
Quick question about the implementation of the option combining, do you think it makes sense to only require the additional flag(eg. -C, -Z, -W), without needing the help subcommand? Basically
rustc --help -C -Z -W instead of
rustc --help -C help -Z help -W help
I just think it will be less verbose and still unambiguous because the --help option specifies a print-and-exit action anyway.
One consideration might be that it may require refactoring the current flag logic to not require a subcommand, which may affect current behavior, but I'm not sure.
There was a problem hiding this comment.
I don’t think that’s worth it, no. That’s ambiguous with a codegen flag named -Z (which doesn’t exist today, but well, seems nicer to not have the ambiguity at all).
I don’t expect this to be very common so I don’t think we need to optimize for brevity.
|
@jkelley129 i don't mind if you use an LLM to generate this code, but please disclose that that's what happening, and please do not use an LLM to generate your responses to my review comments. |
|
I am generating most of the code with an LLM, but I can assure you that I am not blindly copying and certainly not using it for review responses. |
|
Just some context: so tbh I don't really care about the rustc man pages personally, since I just don't use them for
In all honesty, if it were just up to me to decide, I would rather just nuke the outdated manpages and just defer to So in that sense, I don't care to block this change simply because I don't care about the manpages. However, specifically @jyn514 when you ask me about
That, properly implemented, would have to be some kind of compiletest capability directive which is fed information from a tool availability discovery in bootstrap. If you want the run-make test to actually faithfully report "ignored" and not "silently green" (which I consider deceptive tbh). Furthermore, I'm not sure if CI has EDIT: I guess it also depends on which team is supposed to be maintaining this.
|
|
Implementation wise, I didn't have time to look deep, but perhaps a potentially more maintainable approach is to:
I would be much less hesistant if the implementation approach adopted was akin to this, because of the auto-bless support, and in that case, I would actually be perfectly fine with making it (the test and the manpage output changes) blocking in CI. But if the formulation is a manual shell script with no autobless, I would be very against a blocking test that tries to expect the manpage differences from |
|
TIL we have a A few questions (not a full review) after a cursory look:
This change adds new dependencies ( |
| @@ -0,0 +1,97 @@ | |||
| #!/bin/bash | |||
There was a problem hiding this comment.
I'm not a shell wizard so I've run this script through shellcheck v0.11 to have a second opinion:
$ shellcheck generate-rustc-man.sh
In generate-rustc-man.sh line 3:
set -euo pipefail
^------^ SC3040 (warning): In POSIX sh, set option pipefail is undefined.
In generate-rustc-man.sh line 9:
trap "rm -f '$INCLUDE_FILE'" EXIT
^-----------^ SC2064 (warning): Use single quotes, otherwise this expands now rather than when signalled.
For more information:
https://www.shellcheck.net/wiki/SC2064 -- Use single quotes, otherwise this...
https://www.shellcheck.net/wiki/SC3040 -- In POSIX sh, set option pipefail ...
|
@jieyouxu if you would like to have a final review pass before that gets approved that seems fine to me, but i think having three people trying to change the design direction does more harm than good. the risks here are low. the man page is already horribly broken. |
|
Fine by me. |
|
Ok, so a clarifying question on where you'd like me to go on this. I think what the idea right now is to instead write a run-make test with |
|
@jkelley129 yes, that's right. see https://doc.rust-lang.org/nightly/nightly-rustc/run_make_support/diff/struct.Diff.html for how to support
hm, ok. there should be docs for that at https://rustc-dev-guide.rust-lang.org/tests/directives.html#adding-a-directive. i would expect it to be called |
10 participants
Summary
This PR introduces a reproducible and automated process for generating the
rustc(1)man page directly from the compiler’s help output, ensuring it always reflects the current CLI.Details
Adds
src/doc/man/generate-rustc-man.sh:A shell script that invokes
help2manon any givenrustcbinary, via that binary’s--help -voutput.src/doc/man/rustc.1Documentation in
src/doc/man/README.md:Rationale
The man page is generated entirely from the compiler’s current help text, eliminating drift between documentation and reality.
The script ensures anyone can regenerate the man page exactly, as long as they have a target compiler binary and help2man installed.
By keeping the script and artifacts local to
src/doc/man, there’s no need to depend on build system internals.Usage
After building
rustc, simply run:help2manon the host system (apt install help2man,dnf install help2man, etc).Notes
Ready for feedback or further suggestions!