Add CI documentation (0xPolygonHermez#949)

* Add CI documentation

* add dependabot

* update image name

.github/workflows/push-docker-develop.yml

@@ -48,4 +48,4 @@ jobs:
           platforms: linux/amd64,linux/arm64
           push: true
           tags: |
-            hermeznetwork/zkprover-mock:latest
+            hermeznetwork/zkprover-mock:develop

docs/ci/README.md

@@ -0,0 +1,7 @@
+# CI documentation
+
+* [GitHub actions](./actions.md)
+* [ok-to-test](./ok-to-test.md)
+* [Groups and parallel execution](./groups.md)
+* [Test operations manager](./opsman.md)
+* [dependabot](./dependabot.md)

docs/ci/actions.md

@@ -0,0 +1,100 @@
+# GitHub Actions
+
+This documents gives a brief overview of the existing GitHub Actions on the repo
+
+## lint
+
+Runs this [golang linter] over the code, this is the [linter configuration file].
+
+### When is executed
+
+PR opened and pushing changes to PRs.
+
+## ok-to-test
+
+Part of our setup for running tests on PRs from forks, you can read more about it
+in the [ok-to-test document].
+
+### When is executed
+
+PR opened and pushing changes to PRs.
+
+## push-docker-develop
+
+Pushes docker images to docker hub, the images pushed are:
+* `hermeznetwork/zkevm-node:develop`
+* `hermeznetwork/zkprover-mock:develop`
+
+### When is executed
+
+Changes pushed to the `develop` branch.
+
+## push-docker
+
+Pushes docker images to docker hub, the images pushed are:
+* `hermeznetwork/zkevm-node:latest`
+* `hermeznetwork/zkprover-mock:latest`
+
+### When is executed
+
+Changes pushed to the `main` branch.
+
+## test-e2e
+
+Runs e2e tests divided in several groups executed in parallel, read more about
+[CI groups].
+
+### When is executed
+
+PR opened and pushing changes to PRs. There are two variants, `trusted` and
+`from-fork`, depending on the procedence of the PR, more about it in the
+[ok-to-test document].
+
+## test-full-non-e2e
+
+Runs all the non-e2e tests.
+
+### When is executed
+
+PR opened and pushing changes to PRs. There are two variants, `trusted` and
+`from-fork`, depending on the procedence of the PR, more about it in the
+[ok-to-test document].
+
+## updatedeps
+
+The `zkevm-node` repo requires some external resources for working. We call
+these resources custom dependencies (as opposed to the golang packages required
+by the code).
+
+The goal of the `updatedeps` action is to keep these custom dependencies up to
+date. It checks the external resources with content required by this repo and
+proposes a PR in case it finds any changes. The code executed can be found in
+the [dependencies package].
+
+Currently we are checking [three types of custom dependencies]:
+* External docker images used in the [docker compose file],
+`hermeznetwork/geth-zkevm-contracts` and `hermeznetwork/zkevm-mock-prover`. For
+each image the code compares the digest existing in the docker compose file with
+the digest returned by docker hub API, if they differ it includes the new one in
+the docker compose file.
+* Protocol buffer files from [comms protocol repo]: after checking the files
+for changes the client/server golang code is generated from them.
+* Test vectors from the [test vectors repo].
+
+With all the potential changes we create a new PR and the tests are run on it,
+so that we can review and eventually approve the changes to be included in the
+`zkevm-node` repo.
+
+### When is executed
+
+It runs as an scheduled action, every 3 hours.
+
+[golang linter]: https://golangci-lint.run/
+[linter configuration file]: ../../.golangci.yml
+[ok-to-test document]: ./ok-to-test.md
+[CI groups]: ./groups.md
+[dependencies package]: ../../scripts/cmd/dependencies
+[three types of custom dependencies]: ../../scripts/cmd/dependencies.go
+[docker compose file]: ../../docker-compose.yml
+[comms protocol repo]: https://github.com/0xPolygonHermez/zkevm-comms-protocol/
+[test vectors repo]: https://github.com/0xPolygonHermez/zkevm-testvectors

docs/ci/dependabot.md

@@ -0,0 +1,7 @@
+# dependabot
+
+We use [dependabot] to keep the golang dependencies up to date. Its settings can
+be found in this [configuration file].
+
+[dependabot]: https://github.com/dependabot
+[configuration file]: ../../.github/dependabot.yml

docs/ci/ok-to-test.md

@@ -0,0 +1,58 @@
+# ok-to-test workflow
+
+By default, when proposing a PR from a fork, the secrets won't be available in
+the actions executed for that PR. This can be problematic if the tests require
+secrets to run, in that case the reviewers can't know if the changes proposed
+will break the tests or not.
+
+## How it works
+In order to solve this issue we integrated the [ok-to-test workflow]. It works
+as follows:
+* The Github actions executed on PRs are split in 3 groups:
+  * actions that don't require secrets.
+  * trusted actions, they require secrets and are executed from branches in the
+  `zkevm-node` repo, secrets are always available to them
+  * from-fork actions, they require secrets and are executed from forks, secrets
+  are available after approval from users with write access to the repo.
+* When a PR is created from a branch in the `zkevm-node` repo, the actions that
+don't require secrets and trusted actions are executed.
+* When a PR is created from a fork, the actions that don't require secrets are
+executed, and the from-fork actions are executed after a user with write access
+comments in the PR `/ok-to-test sha=<commit sha>` with the sha of the commit over
+which the actions should run. Before adding this comment the user should have
+reviewed the code and verified that it won't suppose a security treat (for
+instance, the reviewer should verify that the PR is not adding new actions, or is
+not trying to disclose the existing secrets).
+
+## Requirements
+Our setup relies on the existence of a repo secret called `PERSONAL_ACCESS_TOKEN`
+with the value of a personal access token with repo access scope.
+
+## How to add the ok-to-test functionality to an existing workflow
+In order to transform an existing wokflow into one that use the ok-to-test
+functionality it should be changed like this:
+* Add the `repository_dispatch` entry like here https://github.com/0xPolygonHermez/zkevm-bridge-service/pull/148/files#diff-107e910e9f2ebfb9a741fa10b2aa7100cc1fc4f5f3aca2dfe78b905cbd73c0d2R9-R10
+* Duplicate the job, if it is called `build`, copy it to `from-fork-build` and
+rename `build` to `trusted-build`.
+* In `trusted-build` add this `if` as the first item:
+```
+if: github.event_name == 'pull_request' && github.event.pull_request.head.repo.full_name == github.repository
+```
+* In `from-fork-build`:
+  * Add this `if` as the first item:
+  ```
+  if:
+      github.event_name == 'repository_dispatch' &&
+      github.event.client_payload.slash_command.sha != '' &&
+      contains(github.event.client_payload.pull_request.head.sha, github.event.client_payload.slash_command.sha)
+  ```
+  * If it has a checkout action, replace it with:
+  ```
+  - name: Fork based /ok-to-test checkout
+    uses: actions/checkout@v2
+    with:
+      ref: 'refs/pull/${{ github.event.client_payload.pull_request.number }}/merge'
+  ```
+  * Add this code as the last step https://github.com/0xPolygonHermez/zkevm-bridge-service/pull/148/files#diff-107e910e9f2ebfb9a741fa10b2aa7100cc1fc4f5f3aca2dfe78b905cbd73c0d2R60-R88
+
+[ok-to-test workflow]: https://github.com/imjohnbo/ok-to-test

docs/ci/opsman.md

@@ -0,0 +1,26 @@
+# Test operations package
+
+We use the functionality provided by the [test operations package] in order to
+manage components used during tests, specially e2e.
+
+The main functionality used currently in the tests is related to managing
+containers, the package exposes the function `StartComponent` which takes a
+container name (as defined in the [docker compose file]) and a variadic parameter
+with a set of condition functions to check when the container can be considered
+as ready. So we can call it without any condition like:
+```
+operations.StartComponent("my-container")
+```
+or adding readiness conditions as:
+```
+operations.StartComponent("my-container", func() (done bool, err error){
+  // run some checks
+  return true, nil
+}, func()(done bool, err error){
+  // run some other checks
+  return true, nil
+})
+```
+
+[test operations package]: ../../test/operations/
+[docker compose file]: ../../docker-compose.yml