Skip to content

Commit

Permalink
mom6-pipeline-run time limit to 50 minutes
Browse files Browse the repository at this point in the history
The MOM6 pipeline is currently timing out due to issues related to the
scheduler, so this patch temporarily increases the limit from 42 to 50
minutes.

The overall timing of these jobs will be recalibrated after the
scheduler has been fixed.
  • Loading branch information
marshallward committed Aug 4, 2021
1 parent b2b153b commit 5e06074
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion tools/MRS/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ mom6-pipeline-run:
time make -f tools/MRS/Makefile.clone .datasets
@echo -e "\e[0Ksection_end:`date +%s`:uncache\r\e[0K"
(echo '#!/bin/tcsh';echo 'time make -f tools/MRS/Makefile mom6-pipeline-run-steps') > job.sh
sbatch --clusters=c3,c4 --nodes=30 --time=0:42:00 --account=gfdl_o --qos=debug --job-name=mom6_examples_tests --output=log.$$CI_PIPELINE_ID --wait job.sh && ( egrep -v '.*: *$$|pagefaults|HiWaterMark=' log.$$CI_PIPELINE_ID ; echo Job returned normally ) || ( cat log.$$CI_PIPELINE_ID ; echo Job failed ; exit 911 )
sbatch --clusters=c3,c4 --nodes=30 --time=0:50:00 --account=gfdl_o --qos=debug --job-name=mom6_examples_tests --output=log.$$CI_PIPELINE_ID --wait job.sh && ( egrep -v '.*: *$$|pagefaults|HiWaterMark=' log.$$CI_PIPELINE_ID ; echo Job returned normally ) || ( cat log.$$CI_PIPELINE_ID ; echo Job failed ; exit 911 )
test -f results.ignore/restart_stats_gnu.tar.gz || ( echo Batch job did not complete ; exit 911 )
@echo -e "\e[0Ksection_start:`date +%s`:cache[collapsed=true]\r\e[0KCaching results"
time tar zvcf $(CACHE_DIR)/results-$(CI_PIPELINE_ID).tgz results.*
Expand Down

0 comments on commit 5e06074

Please sign in to comment.