You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As the common case where multiple workers are submitted to be stopped is just submitting all workers to stop, it would be good to special case this and pass JOB_IDS_SESSION_ALL to DRMAA. This would bypass the need for the loop.
There are a few options:
Detect if all workers were passed in (easy, maybe a little hacky)
Make the worker_ids parameter optional and the default value equal to all workers (easy, straightforward)
Integrate this into close (easy, hacky without 1 or 2)
Background the task somehow using threads, dask.delayed, or similar (may need a few tricks depending)
In cases where there are lots of workers
stop_workersis slow. This appears to be caused by closing each worker in afor-loop.As the common case where multiple workers are submitted to be stopped is just submitting all workers to stop, it would be good to special case this and pass
JOB_IDS_SESSION_ALLto DRMAA. This would bypass the need for the loop.There are a few options:
worker_idsparameter optional and the default value equal to all workers (easy, straightforward)close(easy, hacky without 1 or 2)Thoughts?