New wrapped CLI using click

[Crivella]

Enabling

Using pip and optional python dependencies:

pip install easybuild-framework[eb_click]

How it works

• Extract all the parameters from EasyBuildOptions and use them to generate a CLI with click
• Uses click + rich_click to have a fancier looking CLI
• click is only used for the nicer help and auto completion, but the parameters are still passed as is to the normal parser

Autocomplete

From click docs add the following to the .bashrc or venv activation script.

eval "$(_EB_COMPLETE=bash_source eb)"

Alternatively see aiida-core EC to activate it as a module

Advantages

• Nicer looking
• Automatic integration and autocomplete with bash zsh fish shells

TODO

• Implement/convert more types for better autocompletion

Maybe

• Add autocomplete for path options (do not think there is a way that they are flagged as such in EB right now)
• Better integrate logging with rich (moved in favor of Add rich colors to print_msg #5125)
• Rewrite the CLI logic to use click natively and divide the commands in subgroups to have an easier to read help (Discussion moved to Modern CLI for easybuild #5122)

How would this look like

eb --help

Error

Normal build run

Opt in/out

the new behavior will only be turned on if click is installed, otherwise the old one will be used.

[boegel]

This looks kind of nice, but I'm not a big fan of introducing a required 3rd party dependency.

Is there any way we can support this opt-in, for example by defining an environment variable like $EB_CLICK_CLI?

[Crivella]

We could make use of
https://setuptools.pypa.io/en/latest/userguide/dependency_management.html#optional-dependencies

probably not only for this but also for all the optional dependencies we specify in
https://docs.easybuild.io/installation/#optional_python_packages

Then we add a check on top of eb2 to try and import the required packages and print an error if they are not there suggesting to install the optional dependencies (if we keep eb2 and eb separate, otherwise we could just fallback to the normal eb behavior if they are not there and potentially also allow to disable the behavior with the env variable)

[Crivella]

@boegel

So for now the eb2 CLI is separate from the existing eb one.
I've added an optional set of install dependencies so that if one does pip install easybuild-framework[eb2] it also install the stuff required (and optional) for the new CLI.
The CLI will still be present if the dependencies are not there but will print an informative message instead of actually running

try:
    import click as original_click
except ImportError:
    def eb():
        """Placeholder function to inform the user that `click` is required."""
        print('Using `eb2` requires `click` to be installed.')
        print('Either use `eb` or install `click` with `pip install click`.')
        print('`eb2` also uses `rich` and `rich_click` as optional dependencies for enhanced CLI experience.')
        print('Exiting...')
        sys.exit(0)

The approach of using an ENV variable i think make sense if we decide to merge eb with eb2 to only have one CLI with both behaviors, but it might be difficult to reconcile the autocomplete as they come from the environment

[Crivella]

Example of a modified EC to test installing easybuild with easybuild with the new CLI

EasyConfig

easyblock = 'EB_EasyBuildMeta'

name = 'EasyBuild'
version = '5.2.0'
version_suffix = '-dev'

homepage = 'https://easybuilders.github.io/easybuild'
description = """EasyBuild is a software build and installation framework
 written in Python that allows you to install software in a structured,
 repeatable and robust way."""

toolchain = SYSTEM

local_ef_commit = 'cf8bbb2e73d5468553ad83a53a52f9de8cbd6cdb'
local_eb_commit = 'develop'
local_ec_commit = 'develop'
sources = [
    {
        'filename': f'easybuild-framework-{local_ef_commit[:8]}.tar.xz',
        'git_config': {
            'url': 'https://github.com/Crivella',
            'repo_name': 'easybuild-framework',
            'commit': local_ef_commit,
        }
    },
    {
        'filename': f'easybuild-easyblocks-{local_eb_commit[:8]}.tar.xz',
        'git_config': {
            'url': 'https://github.com/easybuilders',
            'repo_name': 'easybuild-easyblocks',
            'commit': local_eb_commit,
        }
    },
    {
        'filename': f'easybuild-easyconfigs-{local_ec_commit[:8]}.tar.xz',
        'git_config': {
            'url': 'https://github.com/easybuilders',
            'repo_name': 'easybuild-easyconfigs',
            'commit': local_ec_commit,
        }
    }
]
checksums = []

click_autocomplete_bins = ['eb2']

# EasyBuild is a (set of) Python packages, so it depends on Python
# usually, we want to use the system Python, so no actual Python dependency is listed
allow_system_deps = [('Python', SYS_PYTHON_VERSION)]

local_pyshortver = '.'.join(SYS_PYTHON_VERSION.split('.')[:2])

sanity_check_paths = {
    'files': ['bin/eb'],
    'dirs': ['lib/python%s/site-packages' % local_pyshortver],
}

moduleclass = 'tools'

Having the following installed on the system python (system- or user-wide) will enable:

• click -> the new CLI (for now actually required to pass the click_autocomplete_bins sanity check)
• rich -> new CLI with tracebacks
• rich_click -> new CLI with rich styling for help and errors

(Ideally this would be added as system packages?

[Crivella]

The current version now does not add a new eb2 CLI anymore but just replaces the easybuild.main inside the eb launcher with easybuild.cli

The function invoked there will either be a wrapper to the old one if click is not installed or the new click-wrapped CLI if click is installed.

Checks based on whether rich and rich-click are available are also added to ensure that their specific functionalities are only enabled if available

NOTE

The multi-path autocompletion enabled by https://github.com/Crivella/easybuild-framework/blob/5439684362e867e9d78de889db1ac4aca2a52ca5/easybuild/cli/options/__init__.py#L62-L109 does not work in Bash when the path separator is:

[boegel]

Did a high-level check and took it for a quick spin with the goal of merging this for the EasyBuild v5.3.0 release, but I'm afraid this needs more love...

1. Copyright/license headers are missing in the new files;

2. eb --version fails when click & co are available (this is a blocker):

   $ eb --version
   
    Usage: eb [OPTIONS] [OTHER_ARGS]...
   
    Try 'eb --help' for help
   ╭─ Error ─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╮
   │ No such option: --version (Possible options: --software-version, --terse, --toolchain-version)                                                                                                                                              │
   ╰─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────╯
   

   CI should have caught this... Can we enhance the existing .github/workflows/end2end.yml CI workflow to also runs those basic end-to-end tests in an environment where click is available?

3. There's no way to opt-out to the old style CLI when click is installed. In fact, I much prefer that the user has to do an explicit opt-in to this new (experimental?) CLI, for example by setting the $EB_CLI_CLICK environment variable.

4. We need an exit strategy for this, since we have plans for a serious overhaul of the eb CLI for EasyBuild 6.0 (ETA spring 2027)...

   What's the role of the click-based CLI going forward? Is it mostly a temporary "toy" to help with exploring out options for a more modern CLI?

I do like that the way in which this is implemented barely touches existing code, which makes me way more comfortable to go ahead and merge this as "alternative" CLI.

[boegel]

With the above in mind, I need to lift this beyond the EasyBuild v5.3.0 release, so changing the milestone...

[boegel]

@Crivella FYI, I've cancelled CI workflows here to give precedence to #5166

[Crivella]

@boegel

1. Done
2. Added it manually, it was not being picked up as in the normal parser it was not added as an EasybuildOption through add_group_parser.
3. Added the EB_CLI_CLICK env variable as suggested. Now the question is whether we want to hard fail if click is not available with EB_CLI_CLICK=1 or revert to the old CLI like we do know (or a middle ground where we add a warning but still revert to the old CLI)
4. I would say it both puts in place the machinery to replace the current wrapper click CLI with a new independent one and also starts people to get used to the new auto-complete and colored CLI. The way we want to phase it out in favor of the new one will depend on how we decide to also play it with the old CLI (eg both in place with a deprecation warning).

Also modified the end2end tests to do

• Everything they already did
• Install deps for venv + create a virtualenv
• Install the checkout branch + the easyblock and configs downloaded for the previous test
• run the same tests + also a grep to check that rich_click is being used in eb --help
• One thing is whether we want to do the actual builds of bzip2 in temporary dirs so we rebuild everything the second time or just run with a --rebuild