Implements missing compacton selection functionality#3796
Implements missing compacton selection functionality#3796keith-turner merged 1 commit intoapache:elasticityfrom
Conversation
Compaction selection functionality that opened files was not implemented because it opens user files in the manager. This change implements the functionality inorder to get integration test passing. There is already an open issue about finding a better way to do this. See apache#3526
|
Would this be another task type for #3801 ? |
Yeah, I think it may. It will be similar to computing splits. Wondering how all of these should return data. If the manager stores results in memory then it has to store the data until the FATE operation runs again and is ready to consume it. Thinking it could write the result to the metadata table instead of storing it in manager memory, maybe something like the following.
|
|
I wonder if we should get #3801 where it needs to be and merged, then implement this in that framework vs merging this to get |
Its a small change. I would like to get the verify sunny build working. Would like to also look into expanding what is in the sunny tests to make sure all major features are well covered (ingest, scan, compact, split, merge, bulk import, etc). |
Compaction selection functionality that opened files was not implemented because it opens user files in the manager. This change implements the functionality inorder to get integration test passing. There is already an open issue about finding a better way to do this. See #3526