Adding possibility to generate a json schema file when calling jsonargparse.ArgumentParser.save

[martinResearch]

🚀 Feature request

Add option to generate a json schema alongside the json file when calling jsonargparse.ArgumentParser.save and link to that schema file in the default config json file using the "$schema" key.

Motivation

When I edit json configuration files, it is error prone. I can do spelling mistakes, I do not know what filed are optional, what the possible values for enums are etc.

Pitch

It would be nice if we could generate a json schema alongside the json file when calling jsonargparse.ArgumentParser.save and link to that schema file in the default config json file using the "$schema" key. This would enable the use of a plugin like Json Schema validator in vs code to get auto-completion and validation directly when editing the configuration file.

[mauvilsa]

Thank you for the proposal. This had been requested in the past in Lightning-AI/pytorch-lightning#17564, though having it in this repo is better.

I agree that it would be nice to have a json schema. But note that it is a rather complex feature, which is unlikely to happen soon. The hardest part is the support of subclasses. There can be a large amount of possibilities which would generate quite a large and complex json schema, with plenty of nested oneOf. It could even be that auto-completion plug ins don't even work with such complex schemas.

Another note is, having a way to generate a json schema sounds good. But this being done by save including a $schema key does not sound like the direction to go.

[martinResearch]

Would it make sense to have a first implementation with limited support for subclasses that would be well documented? It would throw an error for unsupported cases. It might be sufficient for a large majority of users?
"But this being done by save including a $schema key does not sound like the direction to go." would be interested to know what you think would be a better alternative.

[mauvilsa]

Currently jsonargparse supports shtab, and to get the tab completions script you do like cli.py --print_shtab bash. A jsonschema feature would be just one more type of completion. I would change this, so it could be for example like cli.py --print_completions shtab-bash and also cli.py --print_completions jsonschema. Note that this allows to get the schema for any CLI. There is no need to implement something specific that calls save. Furthermore, a schema is needed/useful before someone ever wants to save.