port, portindex: add portgroups as a searchable field and selector

[herbygillot]

Enable searching, filtering, and displaying ports by which PortGroups they use. The portgroups field was already populated during port parsing but was not stored in the PortIndex.

• Add portgroups to PortIndex keepkeys, stripping filepath from entries
• Add portgroup:/portgroups: pseudo-portname selector
• Add --portgroup/--portgroups options to port search and port info
• Show Port Groups in default port info output
• Strip filepath from portgroups in port info display (live-parsed data includes filepath, but it is not relevant for display)
• Add mportsearch_portgroups unit test
• Document portgroup selector and options in man pages
• Document previously undocumented selectors: depends_*, replaced_by, subport/subports, subportof

Closes: https://trac.macports.org/ticket/51092

[herbygillot]

@jmroot, @ryandesign: this feature would require a full reindex of the PortIndex in order to deliver accurate results.

How is that normally handled as part of a new release?

[herbygillot]

I'd like to merge this if there are no objections - will leave open for a few days as I didn't give enough time for review for the previous PR I merged in.

[jmroot]

There is a lot of information in a Portfile that is not included in the PortIndex. Each new field increases the index size and slows down searches. What makes this in particular worth the cost?

[pmetzger]

Machines have enough storage that I doubt that anyone will notice the increase in the size of the index, and the slowdown is unlikely to be noticed on a modern Mac either.

[herbygillot]

I measured the actual PortIndex sizes before and after this change, indexing the entire ports tree at the time of this writing with portindex -f:

Before this change:
-rw-r--r--@ 1 herby  staff  25153880 May 14 13:42 PortIndex
-rw-r--r--@ 1 herby  staff    957166 May 14 14:00 PortIndex.quick

After this change:
-rw-r--r--@ 1 herby  staff  26664943 May 14 13:42 PortIndex
-rw-r--r--@ 1 herby  staff    958596 May 14 13:48 PortIndex.quick

So a difference of about a megabyte. There is effectively no cost this day in age.

Being able to search by portgroup is useful for both users and maintainers, and it's a core grouping facet that we can add essentially for free.

Portgroups bring a lot of context as they let us select for language families (Go, Rust), software sources (Github, Codeberg), build frameworks (Cmake, Meson) and more.

[jmroot]

Being able to search by portgroup is useful for both users and maintainers, and it's a core grouping facet that we can add essentially for free.

Up to this point, portgroups have been an implementation detail. I don't think documentation aimed at end users mentions them at all. What is a real world example of something that an end user has wanted to do which this change would make possible?

For maintainers, the main use case I'm aware of is when you are going to make an incompatible change to a portgroup and need to update all portfiles using it. But you want to find portgroup inclusions inside non-default variants, which the PortIndex cannot do, so to find all uses you have to use a full text search anyway.

@raimue Is there anything you can add about the rationale as the author of the original ticket?

[herbygillot]

Up to this point, portgroups have been an implementation detail. I don't think documentation aimed at end users mentions them at all. What is a real world example of something that an end user has wanted to do which this change would make possible?

For maintainers, the main use case I'm aware of is when you are going to make an incompatible change to a portgroup and need to update all portfiles using it. But you want to find portgroup inclusions inside non-default variants, which the PortIndex cannot do, so to find all uses you have to use a full text search anyway.

I think it's worth merging in for purely the maintainer utility alone, but users can use it to locate software families.

If a user wanted to find all Go software in MacPorts, we might have to tell them to use port search --depends_build go or we could tell them to use port search --portgroup golang.

What I think this sheds some light on is that maybe it would be useful for us to have the concept of labels.

Maintainers could add arbitrary labels that get indexed and users can use that to search. Perhaps you could have portgroups set labels (label-append) for ports they manage.

[jmroot]

I think it's worth merging in for purely the maintainer utility alone

Sorry to keep harping on this, but what is the utility? I don't think it's an unreasonably high bar to name something that this does better than existing solutions, but so far nobody has mentioned a single one.

I'm not opposed to doing this if there are good use cases, but I am opposed to doing it for its own sake.

What I think this sheds some light on is that maybe it would be useful for us to have the concept of labels.

Maintainers could add arbitrary labels that get indexed and users can use that to search. Perhaps you could have portgroups set labels (label-append) for ports they manage.

How would this differ from categories?

[pmetzger]

Sorry to keep harping on this, but what is the utility?

What's the harm?

[mohd-akram]

This would help with subports, but as @jmroot mentioned, not with variants. Port groups are also easy to grep because they're on one line (unlike dependencies which require context). We probably do need a dedicated tool for advanced querying. @jmroot previously shared an example that can handle any variable, as not everything can be added to the index. Maybe the documentation updates should be a separate PR? Many things are missing from the man pages so they are sorely needed.