Job.chain() does not work when chaining happens through datasets

[pabloyoyoista]

Even though the job has that information (and is for example showed in the dashboard), job.chain will not return a chain if the datasets are the ones that are chained. This is inconvenient, because standard methods just use datasets as input, thus "breaking" the logic of the job chaining.

[drougge]

Isn't job.dataset().chain() what you want? Is this maybe a documentation issue?

[pabloyoyoista]

Isn't job.dataset().chain() what you want? Is this maybe a documentation issue?

By your answer I assume it's either a documentation or communication issue, yes. We have of course gone back to using job.dataset().chain(). But then I have two wonders:

• What purpose does job.chain() serve? I'd expect two methods that have the same interface in two different objects to behave in an equivalent way.
• Then the board draws graphs of jobs based on their dataset dependencies? Why is that exactly? Wouldn't it make more sense to propagate those dependencies "up" from datasets to jobs, but now "down"?

Probably I just fail to understand how can a dataset have a "previous", and the job that contains it be agnostic to that (when it certainly has a dependency on the job that generated the "previous" of its own dataset).

[pabloyoyoista]

Our conclusion from the discussion today:

• The purpose of job.chain() is to be used when jobs don't have datasets, a.k.a they work with raw files (kinda the use-case we had in ACHT)
• The board is being a bit confusing in this situation, so we don't know exactly what to do

We should document the first thing, but it needs #12