Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

POC for letstest redesign #48

Open
wants to merge 37 commits into
base: master
Choose a base branch
from
Open

POC for letstest redesign #48

wants to merge 37 commits into from

Conversation

adferrand
Copy link
Owner

@adferrand adferrand commented Apr 30, 2019

Some explanations of what is done here:

  • the general concept of letstest is kept: launching several bash scripts on different OS environments to check that certbot-auto, certbot and the plugins are behaving correctly
  • letstest here is a part of certbot_integration_tests, and so takes advantage of the existing logic of certbot-ci: an ACME server (pebble or boulder) is deployed on demand using Docker, and HTTP proxies + DNS are set up to allow parallel executions of tests on this ACME server
  • bash scripts are in asserts/farm_tests: they will be executed on each target environment, and have access to several environment variables to be able to connect to the ACME server
  • the target environments are Docker, specifically set up to mimic as close as possible complete environments: it consists essentially in having a full systemctl up and running, in order to make real modifications in the target environment using apache or nginx plugin
  • the target environments are defined as Dockerfile in assets/farm_envs: it allows specific customization depending of the target OS (centos, fedora, ubuntu...), and they are built before the tests start
  • pytest executes in parallel over the tests and the target environments, based on the content of assets/farm_tests and assets/farm_envs respectively: there is a tox target to test that: tox -e letstest
  • all this logic is tied together using fixtures and parameterized tests functions in a pytest-style module, taking advantages of the same specificities than in other integration tests (reporting, output capturing, CLI params, and so on)

@adferrand adferrand changed the title Poc letstest redesign POC for letstest redesign May 2, 2019
@adferrand adferrand force-pushed the poc-letstest-redesign branch from 8bc9d49 to cf132f7 Compare May 6, 2019 17:59
@adferrand adferrand force-pushed the poc-letstest-redesign branch from fdbdc43 to 72655ce Compare May 7, 2019 08:24
@adferrand adferrand force-pushed the poc-letstest-redesign branch from 72655ce to 0760e2e Compare May 7, 2019 08:40
@adferrand adferrand force-pushed the poc-letstest-redesign branch from 7be5ccc to 518df0b Compare May 7, 2019 12:09
@adferrand adferrand force-pushed the master branch 2 times, most recently from 4ad1784 to add90ce Compare June 14, 2019 14:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant