Add PackageCache support for base packages#1166
Conversation
| // The `base` package itself has no NAMESPACE, for now we generate an empty | ||
| // NAMESPACE, but eventually we will want to fully populate it with a | ||
| // pseudo-NAMESPACE. | ||
| if package.name() == "base" { | ||
| std::fs::write(destination_lock.parent().join("NAMESPACE"), "")?; | ||
| crate::fs::set_readonly(destination_lock.parent().join("NAMESPACE"))?; |
There was a problem hiding this comment.
Good enough for the moment, but we will need to fake it somehow. I couldn't come up with a solid plan for this yet.
|
|
||
| for mirror in mirrors { | ||
| let url = format!("{mirror}/{suffix}"); | ||
| let response = reqwest::blocking::get(&url)?; |
There was a problem hiding this comment.
The download of the R tarball from r-project.org can be somewhat slow, normally taking around 10 seconds for me.
It seems to be faster from the rstudio.com mirror (I tried switching the ordering so RStudio's mirror comes first).
Note that if we hit 30 seconds for the download time, then reqwest will cancel the download and return an error here. I have seen that happen exactly one time. We may want to consider that case in more detail. I'm not sure what the right thing to do is here, maybe try to up it to 1 minute? Much longer than that would be a horrible user experience.
There was a problem hiding this comment.
Update: We have a 10 second timeout on connection and a 40 second timeout for the download. If that times out we try the other CRAN mirror (which can be much faster, as mentioned above)
| pub(crate) struct InstalledPackage { | ||
| key: String, | ||
| name: String, | ||
| library_path: PathBuf, | ||
| description: Description, | ||
| description_hash: String, | ||
| } |
There was a problem hiding this comment.
Some details on how to create this struct have been extracted out because for the base package download I have to create one of these for each of the base packages.
DavisVaughan
force-pushed
the
oak/base-package-cache
branch
from
05fd708 to
d49ea97
Compare
1 participant
The base packages aren't in the typical location, i.e. not in https://cran.r-project.org/src/contrib/.
Instead you must download the whole R tarball located at https://cran.r-project.org/src/base/R-4/, dig into it at
src/library/{base_package}/Rand copy over theR/sources from there.Because the R tarball is a whopping ~40mb, if we detect that we need to do this for any base package, then we go ahead and cache all ~14 base packages.
As a side note, Recommended packages like MASS are in the standard download and
Archive/locations, so we already supported those.