BUG: forcefields add-on has conflicting versions

[computron]

The bug is described towards the end of this discussion:
#1438

Briefly- the forcefields add-on in atomate2 leads to dependency conflicts:

ERROR: Cannot install atomate2 because these package versions have conflicting dependencies.

The conflict is caused by:
    mace-torch 0.3.3 depends on e3nn==0.4.4
    mattersim 1.2.1 depends on e3nn>=0.5.0
    mace-torch 0.3.3 depends on e3nn==0.4.4
    mattersim 1.2.0 depends on e3nn>=0.5.0
    mace-torch 0.3.3 depends on e3nn==0.4.4
    mattersim 1.1.2 depends on e3nn>=0.5.0
    mace-torch 0.3.3 depends on e3nn==0.4.4
    mattersim 1.1.1 depends on e3nn>=0.5.0
    mace-torch 0.3.3 depends on e3nn==0.4.4
    mattersim 1.1.0 depends on e3nn>=0.5.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip attempt to solve the dependency conflict

ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

Suggested resolution from my side is for this add-on to only support a few core forcefields, such as those supported by matgl, and have the user be responsible for adding on more forcefields that they choose.

[computron]

With the proposed PR, which FFs are actually usable when installing the forcefields add-on?

I would still suggest keeping matgl and can talk more about that however I want to first get a sense of what a user would be able to run if they just did pip install atomate2[forcefields]. The goal from my viewpoint is that if they include the add-on they could at least run basic force fields that are close to the up-to-date models. Otherwise, I don't really see the point of even having the add-on if it's not practically useful.

[computron]

Re-opening as my comment about matgl (mentioned twice) was not addressed, and instead it was unilaterally decided to only support CHGNet when installing forcefields. The only justification given was "no favoritism" with no discussion from other devs.

Arguments against "favoritism" argument to only install CHGNet with forcefields add-on:

• atomate2 already plays favorites - VASP is heavily favored over other codes - and VASP is even closed source/commercial. The point of atomate2 is to expose useful functionality for users, not to ensure 100% even coverage of every code and model.
• I don't see how installing matgl, which supports multiple force fields (including those not developed by the matgl team), is more "favoritism" than only installing a single model CHGNet (which is also developed by MP collaborators but gives zero choice to use other models not developed by MP collaborators). The current model seems to heavily play favorites to CHGNet - the user installs the forcefield add-on, and only CHGNet is available. Installing matgl gives users the choice to use other models. It seems to me that installing matgl (and requiring users to install the MP-collaborative CHGNet) is the less favoritism solution.
• The library of models supported by matgl are much more useful to the user than CHGNet. We have done testing on models for many applications and CHGNet is not SOTA. In addition to internal non-public testing, see: https://matbench-discovery.materialsproject.org

The only real argument I see to not install matgl for users is to keep installation stable with the forcefields add-on. My sense however is that since atomate2 already depends on pymatgen, and pymatgen and matgl are heavily aligned, and also matgl is kept up to date with dependencies, that this shouldn't be an issue. If matgl conflicts did become an issue I'd be more supportive of not installing matgl with the forcefields add-on.

[computron]

Btw, my suggestion is to have a forcefields-base that just installs the necessary ase dependency to support force fields and no specific force fields. The forcefields add-on should install useful forcefields supported by matgl. \

The docs would then not say this:

"We have made this choice both to avoid the appearance of favoritism (both chgnet and atomate2 are Materials Project-supported projects), and to avoid dependency conflicts between MLFF packages"

and would instead read:
"The forcefields-base add-on only installs the required infrastructure to run force fields. Because many force fields have conflicting dependencies, the user is responsible for installing their force fields of choice. In contrast, the forcefields add-on also installs the matgl library which includes many common (but not all) force fields. Note that this may introduce version conflicts with specific force field libraries."

[computron]

Finally, it turns out that for my system, the current forcefields install is not even allowed due to CHGNet requirements ...

>> pip install --upgrade atomate2[forcefields,phonons]==0.1.1
...
...
ERROR: Could not find a version that satisfies the requirement torch>=2.4.1 (from chgnet) (from versions: 2.0.0, 2.0.1, 2.1.0, 2.1.1, 2.1.2, 2.2.0, 2.2.1, 2.2.2)
ERROR: No matching distribution found for torch>=2.4.1

whereas matgl installs fine. This is likely specific to my older system but at least to me matgl is not any less stable than just CHGNet alone.

[JaGeo]

I am happy with forcefields-base.

My personal opinion would be that we install a package in forcefields that is compatible with as many forcefields as possible. If this is true for matgl, then why not.... However, usually pytorch and related packages lead to the conflicts.

Alternatively, we call it forcefields-demo or similar to clarify that users should install their fav forcefield if they want to do production work or that they should reflect a bit more on this.

[esoteric-ephemera]

The reason I opted not to include matgl is because to use half of the models in it, one needs dgl, which no longer releases to pypi for mac/windows. That locks the user into much older dependencies: torch==2.2.1 vs. the current 2.11.0.

The only justification given was "no favoritism" with no discussion from other devs.

Happy to hear anyone's thoughts if they feel I excluded them. I simply tried to move quickly to patch a few core workflows, given the "urgency" there.

I really don't get the sense that people expect/want to install a forcefields add-on and get a lot of forcefields. Most would just install the forcefield libraries as they need them

Re favoritism: I'm not a maintainer of matgl but use of the MatPES models in it would obviously render a benefit to me as a coauthor of that work. I picked chgnet as it has minimal dependency overhead, is open source MP software, and has no COI from the other maintainers

[JaGeo]

Maybe we should go with forcefields-demo and forcefield-base then?

Just to clarify: i meant we should install one forcefield package that is compatible with many other forcefields to avoid confusion for users.

I personally do not have much of an opinion on the favorism bit: i don't think our default will influence too much. I think the benchmarks influence much more when it comes to usage.

In any case, @esoteric-ephemera , thanks for taking care of it so quickly.

[computron]

Maybe we should go with forcefields-demo and forcefield-base then?

This is OK with me. From my side I will probably just instruct people to install forcefields-base and then just have them install matgl afterwards.

[esoteric-ephemera]

Sure thing and sounds good to me. If #1455 looks good to everyone, I'll merge that in

[JaGeo]

Fine with me. And thanks!

[computron]

In this case, slight preference for just removing

forcefields-base = ["atomate2[ase]"]

altogether. The docs can just tell the user to install atomate2[ase].

Basically:

• when "forcefields" add-on installed the force fields I needed, it was nice to have
• now that the forcefields-base add-on does not do anything different than the ase add on, I don't really need it