Fix staging and resume for typed record inputs with nested Path fields#7083
Open
stekaz wants to merge 1 commit intonextflow-io:masterfrom
Open
Fix staging and resume for typed record inputs with nested Path fields#7083stekaz wants to merge 1 commit intonextflow-io:masterfrom
stekaz wants to merge 1 commit intonextflow-io:masterfrom
Conversation
✅ Deploy Preview for nextflow-docs-staging canceled.
|
Member
|
Is this related to #7032 ? |
8983f0d to
6e13a14
Compare
Signed-off-by: Stephen Kazakoff <sh.kazakoff@gmail.com>
Contributor
Author
|
Yes - I think they're both in the same problem area, but the failure modes are different. This PR fixes the input-side staging/localization (and resume) for typed records with nested Path fields. The other issue occurs when a record input is reused in an output record causing the downstream process to receive paths that are only valid in the upstream context. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Typed process inputs using record types with nested
Pathfields are (currently) not always handled like normal file inputs.For example, given an input such as:
where
SampleRecord.pathis aPath, the compiler should implicitly stagesample.pathinto the task work directory and rewrite references such as:${resource.path}to the process working directory, for example:
But in some cases this does not happen. The compiler can miss the implicit file stager for nested Path fields. When that happens, task scripts either render the original source path instead of the staged task-local path, or the expected file could be missing from the working directory. And
-resumecould behave incorrectly.