MB-3685 | misc small cypress improvements :) (#4623)

- add `yarn.lock` to cypress folder
- add cypress folder caching to circleCI cache flow
  - this has minimal speed gains, but any gain is a win at this point
- add cypress dep to root dev dependencies
  - this allows for root level opening of cypress test runner via yarn, which IMO is a huge win
  - it's much faster to use the test runner against already running docker pool rather than do a clean test/rebuild
    - __for local dev, unblocking fast test iteration > having a clean test environment for every cypress test run__
- add some dangerfile checks to ensure we're keeping cypress deps in sync
- update instructions in README around running e2e tests
  - call out more explicitly why `run_e2e_tests` may behave unexpectedly

Jira story
- https://dp3.atlassian.net/browse/MB-3685

.circleci/config.yml

@@ -481,6 +481,32 @@ jobs:
             - ~/transcom/mymove/node_modules
       - announce_failure
 
+  pre_deps_cypress:
+    executor: mymove_medium
+    steps:
+      - checkout
+      - restore_cache:
+          keys:
+            - v1-cache-cypress-yarn-{{ checksum "cypress/yarn.lock" }}
+      - restore_cache:
+          keys:
+            - v1-cache-cypress-node-modules-{{ checksum "cypress/yarn.lock" }}
+      - run:
+          name: Install YARN dependencies
+          # setting network concurrency to 1 to follow the others; this is pretty standard for cloud yarning. see note in pre_deps_yarn for more detail.
+          command: yarn install --network-concurrency 1
+      # `v1-cache-cypress-yarn-{{ checksum "cypress/yarn.lock" }}` is used to cache yarn sources
+      - save_cache:
+          key: v1-cache-cypress-yarn-{{ checksum "cypress/yarn.lock" }}
+          paths:
+            - ~/.cache/cypress/yarn/v1
+      # `v1-cache-cypress-node-modules-{{ checksum "cypress/yarn.lock" }}` is used to cache installed node modules
+      - save_cache:
+          key: v1-cache-cypress-node-modules-{{ checksum "cypress/yarn.lock" }}
+          paths:
+            - ~/.cache/cypress/node_modules/v1
+      - announce_failure
+
   # `check_generated_code` is used to ensure generated code doesn't change
   check_generated_code:
     executor: mymove_medium
@@ -656,6 +682,12 @@ jobs:
       - restore_cache:
           keys:
             - v3-mymove-node-modules-{{ checksum "yarn.lock" }}
+      - restore_cache:
+          keys:
+            - v1-cache-cypress-yarn-{{ checksum "cypress/yarn.lock" }}
+      - restore_cache:
+          keys:
+            - v1-cache-cypress-node-modules-{{ checksum "cypress/yarn.lock" }}
       - attach_workspace:
           at: /home/circleci/transcom/mymove/bin
       - run: rm -f pkg/assets/assets.go && make pkg/assets/assets.go
@@ -1499,10 +1531,13 @@ workflows:
           requires:
             - pre_deps_golang
 
+      - pre_deps_cypress
+
       - integration_tests:
           requires:
             - pre_deps_golang
             - pre_deps_yarn
+            - pre_deps_cypress
             - check_generated_code
             - push_app_legacy
             - push_app_stg

.gitignore

@@ -11,6 +11,7 @@
 /coverage
 coverage.out
 __snapshots__
+cypress/cypress
 
 # production
 /build

README.md

@@ -58,6 +58,7 @@ in the [LICENSE.txt](./LICENSE.txt) file in this repository.
   * [Test Data Generator](#test-data-generator)
   * [API / Swagger](#api--swagger)
   * [Testing](#testing)
+    * [Troubleshooting tips -- integration / e2e tests](#troubleshooting-tips----integration--e2e-tests)
   * [Logging](#logging)
     * [Log files](#log-files)
   * [Database](#database)
@@ -426,10 +427,25 @@ There are a few handy targets in the Makefile to help you run tests:
 
 * `make client_test`: Run front-end testing suites.
 * `make server_test`: Run back-end testing suites. [Additional info for running go tests](https://github.com/transcom/mymove/wiki/run-go-tests)
-* `make e2e_test`: Run e2e testing suite. Note: this will not necessarily reflect the same results as in the CI environment, run with caution.
+* `make e2e_test`: Run e2e testing suite.
+  * Note: this will not necessarily reflect the same results as in the CI environment, run with caution. One of the reasons for this is it's pulling actual cypress latest, which as of this writing is `5.0.0`. Another reason is your `.envrc` is going to populate your dev environment with a bunch of values that `make e2e_test_docker` won't have.
+  * Note also: this runs with a full clean/rebuild, so it is not great for fast iteration. Use `yarn test:e2e` when working with individual tests
+* `yarn test:e2e`: Open the cypress test runner against your already running servers and inspect/run individual e2e tests. (Should better reflect CI environment than above, but not as well as below.)
+  * Note: You must already have the servers running for this to work! This may not reflect the same results as CI for the same reason as the above re: `.envrc` values. However, it is __significantly__ faster because you can run individual tests and not have to deal with the clean/rebuild.
+* `yarn test:e2e-clean`: Resets your dev DB to a clean state before opening the Cypress test runner.
 * `make e2e_test_docker`: Run e2e testing suite in the same docker container as is run in CircleCI.
+  * Note: this also runs with a full clean/rebuild, so it is not great for fast iteration. Use `yarn test:e2e` when working with individual tests.
 * `make test`: Run e2e, client- and server-side testing suites.
 
+#### Troubleshooting tips -- integration / e2e tests
+
+When running locally, you may find that retries or successive runs have unexpected failures. Some of the integration tests are written with the assumption that they will only be run against a clean DB. If you're working with one of these and don't have time to fix them to properly set up and clean up their state, you can use this command to reset your local dev db before opening the test runner. Note that if you choose not to fix the offending test(s), you'll have to repeatedly close the test runner to re-clean the the DB. You won't be able to take advantage of Cypress's hot reloading!
+
+If you suspect memory issues, you can further inspect this with the commands:
+
+* `yarn test:e2e-debug`, which runs `yarn test:e2e` with DEBUG stats
+* `yarn test:e2e-debug-clean`, which runs `yarn test:e2e-clean` with DEBUG stats
+
 ### Logging
 
 We are using [zap](https://github.com/uber-go/zap) as a logger in this project. We currently rely on its built-in `NewDevelopment()` and `NewProduction()` default configs, which are enabled in any of the executable packages that live in `cmd`.

cypress.json

@@ -0,0 +1 @@
+{}

cypress/Dockerfile.cypress

@@ -1,7 +1,8 @@
 FROM cypress/base:12
 
 COPY package.json /package.json
-RUN npm install --dev --silent
+COPY yarn.lock /yarn.lock
+RUN yarn install --dev --silent
 
 COPY . /cypress
 COPY cypress.json /cypress.json

cypress/support/constants.js

@@ -2,7 +2,7 @@ export const longPageLoadTimeout = 10000;
 export const fileUploadTimeout = 10000;
 
 // Base URLs
-const cypressClientPort = 4000; //change this to 3000 to run cypress against dev instance
+let cypressClientPort = Cypress.env('testClientPort') || 4000; //change this to 3000 to run cypress against dev instance
 export const milmoveBaseURL = `http://milmovelocal:${cypressClientPort}`;
 export const officeBaseURL = `http://officelocal:${cypressClientPort}`;