Tracking issue: man pages

[yoshuawuyts]

This is a tracking issue for man page creation. The initial goal is to provide a flow to generate man pages from Structopt instances. This is a specialization of https://github.com/rust-lang-nursery/cli-wg/issues/23.

Example

The goal is to be able to use code similar to this.

// src/cli.rs
extern crate structopt;

#[derive(StructOpt, Debug)]
pub struct Opts {
  /// TCP port to listen to.
  #[structopt(
    short = "p", long = "port", env = "PORT", default_value = "8080"
  )]
  port: usize,
}

// src/lib.rs
#[macro_use]
extern crate structopt;

pub mod cli;

// build.rs
extern crate structop_to_man;

use structopt::{clap::Shell, StructOpt};

include!("src/lib.rs");

fn main() {
  let outdir = ::std::env::var_os("OUT_DIR").expect("OUT_DIR not found.");
  let mut app = cli::Opts::clap();
  app.gen_completions("my_app", Shell::Fish, &outdir); // generate shell completions
  structop_to_man::gen_man(&app, &outdir); // generate man pages
}

Checklist

I think the following steps are required to make this happen.

• Base man page generator (roff) - https://github.com/killercup/roff-rs
• Higher level man page generator (wrap roff, write to Writer).
• Parse Clap struct & convert to individual fields (see Question: how to extract {flags, opts, subcommands, ...} clap-rs/clap#1285 for current status).
• Combine Clap struct parser + man page generator into single crate that can generate a man page and write it to Writer.

[yoshuawuyts]

Note to self: the best way to approach this would be to work on the clap v3 branch, because it has an actual API intended to expose flags & stuff.

[yoshuawuyts]

Cool tool linked from Gitter: http://www.snake.net/software/troffcvt/tbl.html.

Log

16:35 <Screwtapello> @waywardmonkeys That's pretty much how roff works, for example. "mdoc" is a high-level, semantic markup for manpages, "man" is lower-level, presentational markup, and eventually it all gets ground up to individual PostScript (or whatever) instructinos.
16:35 <waywardmonkeys> @epage And that’s useful because at the highest level, you essentially have your data objects that you want to output .. so transitioning from that to the output is a lot like a Display trait or Debug trait, but instead of going directly to a stream, you’re going to the next level down of IR.
16:37 <Screwtapello> (for instance, roff itself doesn't support table layout; `tbl` takes a semantic description of a table and spits out low-level roff instructions to make it put text in the right places)

[aoikonomopoulos]

Just some things I asked about in the gitter chat.

1. Would it be safer to work with clap-independent data types? I.e. define something to the effect of

struct MinimalArg {
    short: Vec<&str>,
    long: Vec<&str>,
}

which can be consumed, produced or both by argument parsing crates and which is good enough for referencing it from manual page (or other documentation) generators.

2. Is there a plan for having making the generated manual pages accessible on the web? I know I'll often need to read through detailed docs to understand whether a given utility will work for my needs. Reading the source might be less than ideal for people who don't know rust. Even more so if most of the manual page text is in one place, but the names of the arguments are being pulled in from other places in the source.

   Would docs.rs be a good place for html versions of the manpages?

3. Another, more minor, issue is that certain projects might elect to only make some arguments available conditionally. For example, if an external program is available and has been verified to work properly; if some required data is there; if the processor supports a certain feature and so on. In that case, extracting the argument names from an App or a Structopt might only provide the arguments associated with the minimal functionality. I suppose one could argue that programs shouldn't do that then. But, when it comes to UI decisions, I'd argue that some flexibility is warranted. That said, whether the extra complexity is worth it would be up for debate.

   One way to achieve this flexibility would be to work with the hypothetical MinimalArg structs and build clap::Args from them (instead of using .short/.long. Then, the code that generates the man page could reference those flags by name, instead of having to dynamically look them up.

4. With regard to packaging, what would be a good way to expose the names of the man pages (and the way to generate them) to external code? Should man pages always be built during cargo build (e.g. with the build.rs method described above) and then a Cargo.toml key would list the names/paths of all manpages? Would a cargo subcommand make more sense?

[BurntSushi]

Using data types to represent arguments is what ripgrep does, and then just reuses them in build.rs to generate a man page. It is considerably easier to do this in a more confined use case though.

[deg4uss3r]

Some thoughts!

1. I like this! Seems like a great idea since you could also access these to make a difference between -help and --help if you wanted to have the option to expand help. Not traditional, but a possibly nice-to-have?

2. I like this a lot too, I went back and forth so much on writing docs for my simple CLI but finally decided not to because they don't fit. Maybe something like man.docs.rs so they are all in one spot and it's more obvious you're looking at the man page.

3. Maybe the best solution for more complicated things is to have a document comment for man pages e.g. \\$ akin to \\\ but will just format as the man page we're trying to get to so you can not use the struct. Allowing people to write their own, or use the struct if they are having a hard time with complicated CLI switch logic.

[aoikonomopoulos]

@BurntSushi, what problems do you see in general usage?

FWIW, I could see both approaches working out

(a) have the argument-parsing code produce/export the MinimalArg. In this case, we would provide an easy way for the manpage-generating code to generate the options section from an object (e.g. clap::App that allows you to iterate over the available arguments. Note that this scenario is the reason I went with MinimalArg above (as opposed to ripgreps "richer" args) as it's sort of a lowest common denominator.

My main reservation regarding this approach is that you need to include what might be significantly extended text, with some kind of markup, interspersed with the code. I guess this will have to happen somewhere, but see below for an alternate route.

(b) have the argument-parsing code consume MinimalArg-style structs (possibly re-exporting them as in (a)). This would be a bit more tedious, as you'd have to manually reference each option, but means you can build up the man page using e.g. https://github.com/rust-clique/man and simply use rust variables to refer to the argument names, which might look better. Totally subjective, of course, and unlikely to be a concern for most projects.

Given that both the man and info formats have been around for quite some time now, I suppose it's safe to go with a "rich" argument (like ripgrep's RGArg) in any case, as it's sort of unlikely that future code would be significantly limited.

[aoikonomopoulos]

@deg4uss3r, thanks for your comments! I'm not sure I follow re: 1. Are you talking about the documentation generation here? What distinction would MinimalArg enable that isn't already possible?

2. Works for me :-) I imagine one would usually get there by following a link from README.md anyway.

3. Hmm. How would you reference the options in the "man" comment though?

[aoikonomopoulos]

Just to avoid this getting lost, some considerations for the information that we'd want to have available for generating parts of the manpage:

• programs might validly have more than one short flag or option (gonna use option from now on) for the same operation, e.g. for backwards compatibility. Those are normally documented adjacent to each other in the man page.
• ditto for long options
• options might appear as -f, -foo, --foo, +N or even foo. Maximal flexibility is not a goal in itself, of course, but it's worth thinking about what's easy to support.
• options might also have a negative form (--[no-]-debug or -n vs -N). Those are typically documented next to one another.

[yoshuawuyts]

Oh also worth mentioning: work on man page generation has started here https://github.com/rust-clique/man.

[yoshuawuyts]

It took me a while to dig this up, but for posterity: PR to add man page generation to clap is here: clap-rs/clap_generate#1