visual.md

id	title
visual	Visual Testing

How does one test if the UI being rendered appears correctly to the users or how to test if each UI element appears in the right position and size? The traditional way to test the UI of the application has always been manually, which is time consuming.

Visual testing with help of CodeceptJS will help in improving such use cases for the QA folks.

By default CodeceptJS uses WebDriver helper and Selenium to automate browser. It is also capable of taking screenshots of the application and this could be used for visual testing.

Currently there are two helpers available for Visual testing with CodeceptJS

Using Resemble helper

Resemble.js is a great tool for image comparison and analysis, which can be used with CodeceptJS

Setup

To install the package, just run

npm install codeceptjs-resemblehelper --save

Configuring

This helper should be added to codecept.conf.js config file.

Example:

{
   "helpers": {
     "ResembleHelper" : {
       "require": "codeceptjs-resemblehelper",
       "screenshotFolder" : "./tests/output/",
       "baseFolder": "./tests/screenshots/base/",
       "diffFolder": "./tests/screenshots/diff/"
     }
   }
}

To use the Helper, users must provide the three parameters:

• screenshotFolder : This will always have the same value as output in Codecept configuration, this is the folder where webdriverIO saves a screenshot when using I.saveScreenshot method
• baseFolder: This is the folder for base images, which will be used with screenshot for comparison
• diffFolder: This will the folder where resemble would try to store the difference image, which can be viewed later.

Usage

Details about the helper can be found on the Github Repo

Base Image is compared with the screenshot image and test results are derived based on the mismatch tolerance level provided by the user for the comparison

Example

Lets consider visual testing for CodeceptJS Home

Feature('To test screen comparison with resemble Js Example test');

Scenario('Compare CodeceptIO Home Page @visual-test', async (I, adminPage) => {
    I.amOnPage("/");
    I.saveScreenshot("Codecept_IO_Screenshot_Image.png");
    I.seeVisualDiff("Codecept_IO_Screenshot_Image.png", {tolerance: 2, prepareBaseImage: false});
});

In this example, we are setting the expected mismatch tolerance level as 2

Base Image (Generated by User)

Screenshot Image (Generated by Test)

Clearly the difference in both the images visible to human eye is the section about Scenario Driven

Diff Image generated by the helper clearly highlights the section which don't match

Failed Test output

To test screen comparison with resemble Js Example test --
  Compare CodeceptIO Home Page @visual-test
    I see Visual Diff "Codecept_IO_Screenshot_Image.png", {tolerance: 2, prepareBaseImage: false}
MisMatch Percentage Calculated is 2.85
  ✖ FAILED in 418ms


-- FAILURES:

  1) To test screen comparison with resemble Js Example test
       Compare CodeceptIO Home Page @visual-test:

      MissMatch Percentage 2.85
      + expected - actual

      -false
      +true

Codeceptjs-resemblehelper basically comes with two major functions

1. seeVisualDiff which can be used to compare two images and calculate the misMatch percentage.
2. seeVisualDiffForElement which can be used to compare elements on the two images and calculate misMatch percentage.

Using Visual Knight

Visual Knight is a SaaS product which strongly supports CodeceptJS with multiple use cases. It provides an user interface to handle mismatches, statistics and more. It was designed to support Designer, Product Owner and other roles which are not familiar with coding and all this tools. All captured images are saved in a secure cloud system to not mess up your git repository.

Setup

Create an account at Visual Knight and install the npm package

npm install @visual-knight/codeceptjs -D

Configuring

{
    "helpers": {
        "VisualKnight": {
            "require": "@visual-knight/codeceptjs",
            "key": "YOUR_API_KEY",
            "project": "YOUR_PROJECT_ID OR PROJECT_NAME"
        }
    }
}

Usage

/**
 * @param testName {string} Is provided to visual knight (must be unique)
 * @param options {ScreenshotOptions} Contains additional settings
 */
I.compareFullpageScreenshot(testName, options)
/**
 * @param testName {string} Is provided to visual knight (must be unique)
 * @param options {ScreenshotOptions} Contains additional settings
 */
I.compareViewportScreenshot(testName, options)
/**
 * @param cssSelector {string} Is provided to visual knight
 * @param testName {string} Is provided to visual knight (must be unique)
 * @param options {ScreenshotOptions} Contains additional settings
 */
I.compareElementScreenshot(cssSelector, testName, options)

/*
ScreenshotOptions {
  hide?: string[] // Array of css selectors which gets hidden by "opacity: 0",
  remove?: string[] // Array of css selectors which gets hidden by "display: none",
  additional?: object // Data is saved as relation to the variation. (Future: can be used for filtering)
}
*/

You can find the latest documentation here CodeceptJS helper page

Example

Lets consider visual testing for CodeceptJS Home

Feature('To test screen comparison with Visual Knight Example test');

Scenario('Compare CodeceptIO Home Page @visual-test', async (I, adminPage) => {
    I.amOnPage("/");
    I.compareFullpageScreenshot("CodeceptIO Home Page")
});

Depending of your configuration this test will fail if no baseline exists and log the link to the image to accept or automatically accept the first run as baseline.

You can accept the first image as baseline automatically via autoBaseline: true default is false