Skip to content

Commit

Permalink
Fail fast in stuck ansible-galaxy-collection
Browse files Browse the repository at this point in the history
This specific integration test gets stuck periodically causing the
Galaxy jobs to be killed on timeout wasting an hour of runtime. The
module that gets stuck waiting on Pulp is an in-test one, called
`setup_collections`. When it works, the task is complete in around 70
seconds but when it doesn't, it just freezes the whole play.

This patch attempts to make it fail faster by putting a reasonable
timeout value of 2 minutes.
  • Loading branch information
webknjaz committed Aug 23, 2022
1 parent 38db9cf commit f1c56e9
Showing 1 changed file with 12 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,10 @@
from ansible.module_utils._text import to_bytes
from functools import partial
from multiprocessing import dummy as threading
from multiprocessing import TimeoutError


COLLECTIONS_BUILD_AND_PUBLISH_TIMEOUT = 120


def publish_collection(module, collection):
Expand Down Expand Up @@ -241,7 +245,14 @@ def run_module():

pool = threading.Pool(4)
publish_func = partial(publish_collection, module)
result['results'] = pool.map(publish_func, module.params['collections'])
try:
result['results'] = pool.map_async(
publish_func, module.params['collections'],
).get(timeout=COLLECTIONS_BUILD_AND_PUBLISH_TIMEOUT)
except TimeoutError as timeout_err:
module.fail_json(
'Timed out waiting for collections to be provisioned.',
)

failed = bool(sum(
r['build']['rc'] + r['publish']['rc'] for r in result['results']
Expand Down

0 comments on commit f1c56e9

Please sign in to comment.