Install system version-aware and PolyChord update

[JesusTorrado]

Making the install function version-aware.

• GitHub downloads write a version.dat file containing version tag.
• install() detects obsolete versions and lists obsolete packages at the end (if no version detected: obsolete)
• install() gets (--)upgrade option. Off by default: don't overwrite possible user-changes.
• non-github: Planck clik
• github assets: Planck native: lite/CamSpec...
• non-github: Bicep (nothing to do)
• early install and version checks at run time for data
• test camspec and unbinned
• CAMB
• CLASS
• PolyChord
  • merge Polychord 1.20.0 #232
  • wait for new minor version? (see Polychord 1.20.0 #232) -- if so, restore version check
  • tests

Closes #197

[codecov-commenter]

Codecov Report

Merging #237 (40b4b51) into master (d945838) will increase coverage by 0.09%.
The diff coverage is 84.00%.

@@            Coverage Diff             @@
##           master     #237      +/-   ##
==========================================
+ Coverage   87.88%   87.97%   +0.09%     
==========================================
  Files          92       92              
  Lines        8335     8386      +51     
==========================================
+ Hits         7325     7378      +53     
+ Misses       1010     1008       -2     

Impacted Files	Coverage Δ
cobaya/samplers/polychord/polychord.py	32.20% <50.00%> (ø)
cobaya/tools.py	88.81% <50.00%> (-0.13%)	⬇️
cobaya/install.py	82.74% <78.84%> (-1.45%)	⬇️
cobaya/likelihoods/base_classes/planck_clik.py	90.86% <92.30%> (+6.20%)	⬆️
.../likelihoods/base_classes/InstallableLikelihood.py	91.42% <95.23%> (+1.23%)	⬆️
cobaya/likelihoods/base_classes/planck_pliklite.py	86.31% <100.00%> (ø)
cobaya/theories/camb/camb.py	91.32% <100.00%> (+0.03%)	⬆️
cobaya/theories/classy/classy.py	87.21% <100.00%> (-0.05%)	⬇️
cobaya/samplers/mcmc/mcmc.py	90.23% <0.00%> (-0.24%)	⬇️
cobaya/mpi.py	91.96% <0.00%> (+0.40%)	⬆️
... and 1 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d945838...40b4b51. Read the comment docs.

[JesusTorrado]

@cmbant What's the purpose of the following line?

cobaya/cobaya/tools.py

Line 234 in 8152a09

import_module(module_name)

It is run under a if allow_external and not component_path, and if it does not raise an exception, return_class will be run below, which calls load_module, which in turns does the exact same import_module call, without any error handling in between, apparently. So why would't the current code,

try:
    import_module(module_name)
except Exception:
    exc_info = sys.exc_info()
else:
    try:
        return return_class(module_name)
    except:
        exc_info = sys.exc_info()

be equivalent to the simpler following one?

try:
    return return_class(module_name)
except:
    exc_info = sys.exc_info()

(probably I am wrong, but would like to understand why)

[cmbant]

I don't recall exactly, but probably to give clearer error message and shorter stack trace. Note exc_info[0] is ModuleNotFoundError line further down.

[JesusTorrado]

I just checked that the code is indeed equivalent if it would raise an exception at import time, except a possible reload, which is not relevant here. I have substituted below inside the get_class code the lines that would actually run up to import_module in both cases:

try:
    import_module(module_name)
except Exception:
    exc_info = sys.exc_info()
else:
    try:
        if name in sys.modules and reload:
            del sys.modules[name]
        return import_module(name)
    except:
        exc_info = sys.exc_info()

I am going to simplify this one in principle, and do some testing

[JesusTorrado]

Also, I think there was no error

[cmbant]

Upgrading old versions I think can fail because this raises an error (old installs don't have .dat files):

            elif opts.get("github_release"):
                try:
                    with open(os.path.join(path, _version_filename), "r") as f:
                        installed_version = version.parse(f.readlines()[0])
                except FileNotFoundError:  # old install: no version file
                    raise VersionCheckError("Could not read current version.")

Trace

`(base) [aml1005@login-e-13 cobaya]$ cobaya-install --upgrade planck_2018_lowl.EE
[install] Installing external packages at '/home/aml1005/git/cobaya/modules'
[install] The installation path has been written into the global config file: /home/aml1005/.config/cobaya/config.yaml

================================================================================
planck_2018_lowl.EE

[install] Checking if dependencies have already been installed...
[install] Version check failed: Could not read current version.
[install] Installing...
[clik] clik module loaded successfully from /rds/user/aml1005/hpc-work/modules/code/planck/code/plc_3.0/plc-3.01/lib/python/site-packages/clik
Traceback (most recent call last):
File "/home/aml1005/git/cobaya/cobaya/likelihoods/base_classes/InstallableLikelihood.py", line 98, in is_installed
with open(os.path.join(path, _version_filename), "r") as f:
FileNotFoundError: [Errno 2] No such file or directory: '/rds/user/aml1005/hpc-work/modules/data/planck_supp_data_and_covmats/version.dat'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/aml1005/git/cobaya/cobaya/install.py", line 215, in install
success = install_this(path=general_abspath, **kwargs_install)
File "/home/aml1005/git/cobaya/cobaya/likelihoods/base_classes/planck_clik.py", line 207, in install
if force or not cls.is_installed(path=path, code=False, data=True):
File "/home/aml1005/git/cobaya/cobaya/likelihoods/base_classes/planck_clik.py", line 171, in is_installed
result &= native.is_installed(**kwargs)
File "/home/aml1005/git/cobaya/cobaya/likelihoods/base_classes/InstallableLikelihood.py", line 101, in is_installed
raise VersionCheckError("Could not read current version.")
cobaya.tools.VersionCheckError: Could not read current version.
[install] ERROR An unknown error occurred. Delete the external packages folder '/home/aml1005/git/cobaya/modules' and try again. Please, notify the developers if this error persists.
[install] ERROR Installation failed! Look at the error messages above. Solve them and try again, or, if you are unable to solve them, install the packages required by this component manually.

================================================================================

• Summary *
  ================================================================================

[install] ERROR The installation (or installation test) of some component(s) has failed:

• likelihood:planck_2018_lowl.EE
  Check output of the installer of each component above for precise error info.

[exception handler] ---------------------------------------

Traceback (most recent call last):
File "/rds/user/aml1005/hpc-work/anaconda2/bin/cobaya-install", line 11, in
load_entry_point('cobaya', 'console_scripts', 'cobaya-install')()
File "/home/aml1005/git/cobaya/cobaya/install.py", line 549, in install_script
"no_set_global", "skip", "skip_global", "debug", "upgrade"]})
File "/home/aml1005/git/cobaya/cobaya/install.py", line 278, in install
bullet + bullet.join(failed_components))
cobaya.log.LoggedError: The installation (or installation test) of some component(s) has failed:

• likelihood:planck_2018_lowl.EE
  Check output of the installer of each component above for precise error info.

---

`