You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There is a disparity between the doc and the implementation on asynchronous job execution. The doc describes an async option whereas the implementation is the command line option --jobs.
In my pipeline I have some steps that are bigger than others. For the smaller steps I want to run drake --jobs=16 but the big steps will run out of memory or fight for CPU (because the steps are themselves multi-threaded).
The docs describe the "async" option and I think it would be very simple and effective to simply allow "-async" to turn off async execution for a step. In that situation I would expect drake to run that step alone and not in parallel with other steps. In order to do so it may have to wait until previous steps are done.
(There are more complicated variations on this where we might have step options like jobs:4, indicating that this step needs/wants 4 job slots, etc. Another possibility would be for drake could make a $JOBS environment variable available to the step representing how many of the command line arg --jobs slots are currently available. But I think a simple -async would be enough for many use cases)
Thoughts? Thanks.
The text was updated successfully, but these errors were encountered:
There is a disparity between the doc and the implementation on asynchronous job execution. The doc describes an
async
option whereas the implementation is the command line option--jobs
.In my pipeline I have some steps that are bigger than others. For the smaller steps I want to run
drake --jobs=16
but the big steps will run out of memory or fight for CPU (because the steps are themselves multi-threaded).The docs describe the "async" option and I think it would be very simple and effective to simply allow "-async" to turn off async execution for a step. In that situation I would expect drake to run that step alone and not in parallel with other steps. In order to do so it may have to wait until previous steps are done.
(There are more complicated variations on this where we might have step options like
jobs:4
, indicating that this step needs/wants 4 job slots, etc. Another possibility would be for drake could make a$JOBS
environment variable available to the step representing how many of the command line arg--jobs
slots are currently available. But I think a simple -async would be enough for many use cases)Thoughts? Thanks.
The text was updated successfully, but these errors were encountered: