Skip to content

Commit

Permalink
Test all on push events to tested non-master branches. (certbot#6741)
Browse files Browse the repository at this point in the history
We always run a full set of CI tests before beginning the release process. The way this would work previously is we would either trigger tests on the `test-everything` branch to run through Travis' web UI or if it was a point release, create a new branch based on `test-everything` but modify `.travis.yml` so the branch that was pulled in to be tested was the point release branch instead of `master`.

This no longer works because the former `test-everything` tests are now only run when Travis automatically runs our tests nightly.

We could create and maintain a separate branch for the purpose of manually running all tests or remove the conditionals from the latest `.travis.yml` file every time before we want to run these tests, but there must be A Better Way™.

This PR makes the change that in addition to running all tests nightly, they would also run on pushes to tested branches other than master. These changes do not affect the tests run on PRs or on commits to `master`.

What is affected is commits to point release branches and branches named `test-*`. (See [.travis.yml](https://github.com/certbot/certbot/blob/2ddaf3db043ea8526ae3f9ab2ef120b194b2e506/.travis.yml#L177) for what branches we run tests on.) Running all tests on point release branches automates the step of running our full test suite before doing a point release.

The changes to `test-*` could be a mixed bag, however, since we switched to travis-ci.com over 3 weeks ago, I'm the only one who has used this functionality and I personally prefer things this way. At the very least, since these branches don't seem to be widely used, I think we can make this change and reevaluate if it becomes a problem.

* Test all on push events to non-master branches.

* Move branches section up.

* expand comment
  • Loading branch information
bmw authored Feb 6, 2019
1 parent 7e6a1f2 commit 2560ef0
Showing 1 changed file with 32 additions and 32 deletions.
64 changes: 32 additions & 32 deletions .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,16 @@ before_script:
- 'if [ $TRAVIS_OS_NAME = osx ] ; then ulimit -n 1024 ; fi'
- export TOX_TESTENV_PASSENV=TRAVIS

# Only build pushes to the master branch, PRs, and branches beginning with
# `test-` or of the form `digit(s).digit(s).x`. This reduces the number of
# simultaneous Travis runs, which speeds turnaround time on review since there
# is a cap of on the number of simultaneous runs.
branches:
only:
- master
- /^\d+\.\d+\.x$/
- /^test-.*$/

matrix:
include:
# These environments are always executed
Expand Down Expand Up @@ -62,93 +72,95 @@ matrix:
- python: "2.7"
env: TOXENV=nginxroundtrip

# These environments are executed on cron events only
# These environments are executed on cron events and commits to tested
# branches other than master. Which branches are tested is controlled by
# the "branches" section earlier in this file.
- python: "3.7"
dist: xenial
env: TOXENV=py37 CERTBOT_NO_PIN=1
if: type = cron
if: type = cron OR (type = push AND branch != master)
- python: "2.7"
env: BOULDER_INTEGRATION=v1 INTEGRATION_TEST=certbot TOXENV=py27-certbot-oldest
sudo: required
services: docker
if: type = cron
if: type = cron OR (type = push AND branch != master)
- python: "2.7"
env: BOULDER_INTEGRATION=v2 INTEGRATION_TEST=certbot TOXENV=py27-certbot-oldest
sudo: required
services: docker
if: type = cron
if: type = cron OR (type = push AND branch != master)
- python: "2.7"
env: BOULDER_INTEGRATION=v1 INTEGRATION_TEST=nginx TOXENV=py27-nginx-oldest
sudo: required
services: docker
if: type = cron
if: type = cron OR (type = push AND branch != master)
- python: "2.7"
env: BOULDER_INTEGRATION=v2 INTEGRATION_TEST=nginx TOXENV=py27-nginx-oldest
sudo: required
services: docker
if: type = cron
if: type = cron OR (type = push AND branch != master)
- python: "3.4"
env: TOXENV=py34 BOULDER_INTEGRATION=v1
sudo: required
services: docker
if: type = cron
if: type = cron OR (type = push AND branch != master)
- python: "3.4"
env: TOXENV=py34 BOULDER_INTEGRATION=v2
sudo: required
services: docker
if: type = cron
if: type = cron OR (type = push AND branch != master)
- python: "3.5"
env: TOXENV=py35 BOULDER_INTEGRATION=v1
sudo: required
services: docker
if: type = cron
if: type = cron OR (type = push AND branch != master)
- python: "3.5"
env: TOXENV=py35 BOULDER_INTEGRATION=v2
sudo: required
services: docker
if: type = cron
if: type = cron OR (type = push AND branch != master)
- python: "3.6"
env: TOXENV=py36 BOULDER_INTEGRATION=v1
sudo: required
services: docker
if: type = cron
if: type = cron OR (type = push AND branch != master)
- python: "3.6"
env: TOXENV=py36 BOULDER_INTEGRATION=v2
sudo: required
services: docker
if: type = cron
if: type = cron OR (type = push AND branch != master)
- python: "3.7"
dist: xenial
env: TOXENV=py37 BOULDER_INTEGRATION=v1
sudo: required
services: docker
if: type = cron
if: type = cron OR (type = push AND branch != master)
- python: "3.7"
dist: xenial
env: TOXENV=py37 BOULDER_INTEGRATION=v2
sudo: required
services: docker
if: type = cron
if: type = cron OR (type = push AND branch != master)
- sudo: required
env: TOXENV=le_auto_xenial
services: docker
if: type = cron
if: type = cron OR (type = push AND branch != master)
- sudo: required
env: TOXENV=le_auto_jessie
services: docker
if: type = cron
if: type = cron OR (type = push AND branch != master)
- sudo: required
env: TOXENV=le_auto_centos6
services: docker
if: type = cron
if: type = cron OR (type = push AND branch != master)
- sudo: required
env: TOXENV=docker_dev
services: docker
addons:
apt:
packages: # don't install nginx and apache
- libaugeas0
if: type = cron
if: type = cron OR (type = push AND branch != master)
- language: generic
env: TOXENV=py27
os: osx
Expand All @@ -157,7 +169,7 @@ matrix:
packages:
- augeas
- python2
if: type = cron
if: type = cron OR (type = push AND branch != master)
- language: generic
env: TOXENV=py3
os: osx
Expand All @@ -166,19 +178,7 @@ matrix:
packages:
- augeas
- python3
if: type = cron



# Only build pushes to the master branch, PRs, and branches beginning with
# `test-` or of the form `digit(s).digit(s).x`. This reduces the number of
# simultaneous Travis runs, which speeds turnaround time on review since there
# is a cap of on the number of simultaneous runs.
branches:
only:
- master
- /^\d+\.\d+\.x$/
- /^test-.*$/
if: type = cron OR (type = push AND branch != master)

# container-based infrastructure
sudo: false
Expand Down

0 comments on commit 2560ef0

Please sign in to comment.