Document how resolver works internally#1794
Conversation
To have a place to direct users asking wrong questions, but also to align "lingo" we use.
| under the License. | ||
| --> | ||
|
|
||
| Maven Artifact Resolver (former Aether) is central piece of Maven, and thus, |
| --> | ||
|
|
||
| Maven Artifact Resolver (former Aether) is central piece of Maven, and thus, | ||
| is many times target of questions and curiosity how it actually works under the hood. |
There was a problem hiding this comment.
delete ", and thus,
is many times target of questions and curiosity how it actually works under the hood."
|
|
||
| Maven Artifact Resolver (former Aether) is central piece of Maven, and thus, | ||
| is many times target of questions and curiosity how it actually works under the hood. | ||
| This document tries to shed some light on this topic, and explain the main concepts |
There was a problem hiding this comment.
shed some light on this topic --> explain how resolver works under the hood
| This document tries to shed some light on this topic, and explain the main concepts | ||
| and building blocks of Resolver. | ||
|
|
||
| One important aspect of Resolver is that itself, alone is "incomplete". Integrating application, like Maven is |
There was a problem hiding this comment.
One important aspect of Resolver is that itself, alone --> Resolver along
|
|
||
| One important aspect of Resolver is that itself, alone is "incomplete". Integrating application, like Maven is | ||
| the one that provides the "glue" (models) and logic how to resolve versions, ranges, build effective models. | ||
| Hence, Resolver itself, alone is unusable. To make it usable, one needs to complement it with models and |
There was a problem hiding this comment.
Hence, Resolver itself, alone is unusable. To make it usable, one needs to complement it with models and
implementations of missing components. Historically, Maven module completing Resolver is the
org.apache.maven:maven-resolver-provider module.
-->
By itself, Resolver is unusable. One needs to complement it with models and implementations of missing components. Historically, the Maven module completing Resolver is
org.apache.maven:maven-resolver-provider.
| In general, steps "dependency graph collection" and "conflict resolution" are performed together, and lingo we use for that operation is "dependency collection". | ||
| The "flattening" and "artifact resolving" are also usually done together, and we use for those the term "artifact resolving". | ||
| To have the story more confusing, when all the steps are performed together is also called "dependency resolution". | ||
| Resolver API reflects this terminology and offers methods doing collection, resolution or both. |
| To have the story more confusing, when all the steps are performed together is also called "dependency resolution". | ||
| Resolver API reflects this terminology and offers methods doing collection, resolution or both. | ||
| * | ||
| * Method `CollectResult collectDependencies(RepositorySystemSession session, CollectRequest request)` performs only the collection step, |
| Resolver API reflects this terminology and offers methods doing collection, resolution or both. | ||
| * | ||
| * Method `CollectResult collectDependencies(RepositorySystemSession session, CollectRequest request)` performs only the collection step, | ||
| as name suggests. Hence, only steps "collection" and "conflict resolution" are performed. |
| * Method `DependencyResult resolveDependencies(RepositorySystemSession session, DependencyRequest request)` performs both | ||
| collection and resolution steps. | ||
|
|
||
| Also, each subsequent step depends on the previous one, for example a "dirty graph" cannot be flattened (due possible cycles). |
| collection and resolution steps. | ||
|
|
||
| Also, each subsequent step depends on the previous one, for example a "dirty graph" cannot be flattened (due possible cycles). | ||
| And many times, what you want depends on your use case. For example, to investigate dependency graph, one may |
There was a problem hiding this comment.
And many times --> Many times
| This document tries to explain how resolver works under the hood, and explain the main concepts | ||
| and building blocks of Resolver. | ||
|
|
||
| Resolver alone is "incomplete". Integrating application, like Maven is |
There was a problem hiding this comment.
"Integrating application, like Maven is
the one that provides the "glue" (models) and logic how to resolve versions, ranges, build effective models."
-->
Integrating applications like Maven provide the "glue" (models) and logic to resolve versions and ranges and build effective models.
| ## Core Concepts | ||
|
|
||
| At the core of Resolver are **artifacts** and **repositories**. An artifact is basically a | ||
| "symbolic coordinate" of some content. Usually it is a JAR, but it can be really anything, as long as it is |
There was a problem hiding this comment.
"An artifact is basically a
"symbolic coordinate" of some content."
This statement seems factually incorrect. An artifact is not a coordinate. It is the bytes.
| with one local repository (usually a directory on local filesystem) and zero or more remote repositories. | ||
|
|
||
| The term "resolving" is a bit overloaded, but in general it involves following steps: | ||
| * **dependency graph collection** builds the "dependency graph" |
There was a problem hiding this comment.
I meant the list in the doc, not this one list item. That is do you want to convert the four item list in these foru paragraphs to an ordered list?
|
|
||
| We call an artifact "resolvable" if it can be resolved from any available (local or remote) repository. To make an artifact | ||
| "resolvable" from the local repository, one needs to "install" it. To make an artifact "resolvable" from a remote repository, | ||
| one needs to "deploy" it (this is over-simplification; publishing is new term, but it also involves deploy step). |
There was a problem hiding this comment.
an over-simplification
I' don't follow "publishing is new term, but it also involves deploy step)"
There was a problem hiding this comment.
In "old times" you did deploy to remote repositories, even Central. But today, you "deploy" ONLY to your private repositories/MRM hosted ones, while to get something to Central you need publishing. Publishing is NOT anymore supported directly by Maven (and is NOT deploy, that is a lifecycle phase involved on each module, overlaid, publishing is very different today).
So "deploy" -> is lifecycle phase, usually involves m-deploy-p, happens per each module (while we do support "at end").
"publish" -> is what one needs to perform to get artifacts available from Central, involves calling (proprietary) APIs, zipping/packaging of all artifacts being published, performing validation, etc. In Maven 3 there is no "logically equivalent" place to perform it, hence it usually needs extra extensions like https://maveniverse.eu/docs/njord/
In Maven 4 there is a new lifecycle after:all that may serve as a hook, but still, handling (proprietary) API and zipping is not handled there either.
|
|
||
| ### Dependency Graph Collection | ||
|
|
||
| Collecting is the first step. Caller usually provides the root artifact along with the set of remote repositories to use. |
There was a problem hiding this comment.
Collection is the first step. The caller
| This is where filtering is applied as well. | ||
|
|
||
| Resolver historically used "pre-order" to flatten the tree into a list, but Resolver 2 offers three strategies: "pre-order", | ||
| "post-order", and "level order" (the default). These strategies are currently fixed, but in the future we may offer a more |
There was a problem hiding this comment.
I suggest deleting "These strategies are currently fixed, but in the future we may offer a more
flexible API for custom flattening strategies." I doubt that will happen, and if it does this can be updated then
|
|
||
| ### Artifact Resolution | ||
|
|
||
| Artifact resolution is the process of resolving (downloading and caching, if needed) the actual artifact payload from |
There was a problem hiding this comment.
perhaps delete "payload". We haven't used the term before, though it might be useful to distinguish the BLOB from the BLOB+coordinates
| * Method `DependencyResult resolveDependencies(RepositorySystemSession session, DependencyRequest request)` performs both | ||
| collection and resolution steps. | ||
|
|
||
| Also, each subsequent step depends on the previous one. For example a "dirty graph" cannot be flattened (due possible cycles). |
There was a problem hiding this comment.
Also, each --> Each
due --> due to
| collection and resolution steps. | ||
|
|
||
| Also, each subsequent step depends on the previous one. For example a "dirty graph" cannot be flattened (due possible cycles). | ||
| Many times, what you want depends on your use case. For example, to investigate dependency graph, one may |
| Also, each subsequent step depends on the previous one. For example a "dirty graph" cannot be flattened (due possible cycles). | ||
| Many times, what you want depends on your use case. For example, to investigate dependency graph, one may | ||
| want to collect the dirty graph, but not conflict resolve it. Moreover, dirty graphs are usually not even resolvable | ||
| due cycles. |
There was a problem hiding this comment.
due to
though this is redundant with above
|
Am merging this page too, but we can continue iterating and improving it later. Is still much better than nothing. |
To have a place to direct users asking wrong questions, but also to align "lingo" we use.