Material-UI Testing

Thanks for writing tests! Here's a quick run-down on our current setup.

Getting started

1. Add a unit test to packages/*/src/TheUnitInQuestion/TheUnitInQuestion.test.js or an integration test packages/*/test/.
2. Run yarn test:watch.
3. Implement the tested behavior
4. Open a PR once the test passes or you want somebody to review your work

Tools we use

• @testing-library/react
• chai
• sinon
• mocha
• karma
• enzyme (old tests only)
• vrtest-mui
• docker
• jsdom

Writing Tests

For all unit tests, please use the return value from test/utils/createClientRender. It prepares the test suite and returns a function with the same interface as render from @testing-library/react.

For new tests please use expect from the BDD testing approach. Prefer to use as expressive matchers as possible. This keeps the tests readable, and, more importantly, the message if they fail as descriptive as possible.

In addition to the core matchers from chai we also use matchers from chai-dom.

Deciding where to put a test is (like naming things) a hard problem:

• When in doubt put the new test case directly in the unit test file for that component e.g. material-ui/src/Button/Button.test.js.
• If your test requires multiple components from the library create a new integration test.
• If you find yourself using a lot of data-testid attributes or you're accessing a lot of styles consider adding a component (that doesn't require any interaction) to test/regressions/tests/ e.g. test/regressions/tests/List/ListWithSomeStyleProp

Unexpected calls to console.error or console.warn

By default our test suite fails if any test recorded console.error or console.warn calls:

The failure message includes the name of the test. The logged error is prefixed with the test file as well as suffixed with the full test name and test file. This should help locating the test in case the top of the stack can't be read due to excessive error messages. The error includes the logged message as well as the stacktrace of that message. Unfortunately the stacktrace is currently duplicated due to chai.

You can explicitly expect no console calls for when you're adding a regression test. This makes the test more readable and properly fails the test in watchmode if the test had unexpected console calls.

Writing a test for console.error or console.warn

If you add a new warning via console.error or console.warn you should add tests that expect this message. For tests that expect a call you can use our custom toWarnDev or toErrorDev matchers. The expected messages must be a subset of the actual messages and match the casing. The order of these message must match as well.

Example:

function SomeComponent({ variant }) {
  if (process.env.NODE_ENV !== 'production') {
    if (variant === 'unexpected') {
      console.error("That variant doesn't make sense.");
    }
    if (variant !== undefined) {
      console.error('`variant` is deprecated.');
    }
  }

  return <div />;
}
expect(() => {
  render(<SomeComponent variant="unexpected" />);
}).toErrorDev(["That variant doesn't make sense.", '`variant` is deprecated.']);

function SomeComponent({ variant }) {
  if (process.env.NODE_ENV !== 'production') {
    if (variant === 'unexpected') {
      console.error("That variant doesn't make sense.");
    }
    if (variant !== undefined) {
      console.error('`variant` is deprecated.');
    }
  }

  return <div />;
}
expect(() => {
  render(<SomeComponent />);
}).not.toErrorDev();

Visual regression tests

We try to use as many demos from the documentation as possible; however, we can't replace one with the other as they address different needs. With the regression tests:

• You might need to test a more complex situation, e.g. a stress test of the grid.
• You might need to test a simpler situation, e.g. a static progress bar.

Commands

Material-UI uses a wide range of tests approach as each of them comes with a different trade-off, mainly completeness vs. speed.

React API level

Run the core mocha unit/integration test suite.

To run all of the unit and integration tests run yarn test:unit

If you want to grep for certain tests add -g STRING_TO_GREP.

Watch the core mocha unit/integration test suite.

yarn test:watch

First, we have the unit test suite. It uses mocha and a thin wrapper around @testing-library/react. Here is an example with the Dialog component.

Next, we have the integration tests. They are mostly used for components that act as composite widgets like Select or Menu. Here is an example with the Menu component.

Create HTML coverage reports

yarn test:coverage:html

When running this command you should get under coverage/index.html a full coverage report in HTML format. This is created using Istanbul's HTML reporter and gives good data such as line, branch and function coverage.

DOM API level

Run the mocha test suite using the karma runner.

yarn test:karma

Testing the components at the React level isn't enough; we need to make sure they will behave as expected with a real DOM. To solve that problem we use karma, which is almost a drop in replacement of jsdom. Our tests run on different browsers to increase the coverage:

• Headless Chrome
• Chrome, Firefox, Safari, and Edge thanks to BrowserStack

Browser API level

In the end, components are going to be used in a real browser. The DOM is just one dimension of that environment, so we also need to take into account the rendering engine.

Run the visual regression tests

yarn test:regressions

Next, we are using docker to take screenshots and comparing them with the baseline. It allows catching regressions like this one:

Here is an example with the Menu component.

Installation

The visual regression tests suite has a hard dependency on docker. You need to install it, then run the following commands:

docker-compose up -d

Due to issues with networking in OS X, getting the container to see the test page may require additional configuration as the docker0 interface does not exist.

You can create an alias for the loopback interface using the instructions provided at https://docs.docker.com/docker-for-mac/networking/#/there-is-no-docker0-bridge-on-macos

sudo ifconfig lo0 alias 10.200.10.1/24

In our vrtest config this is set as the default, although it can be overridden with an env var:

testUrl: process.env.DOCKER_TEST_URL || 'http://10.200.10.1:3090',

In addition to docker, the visual regression tests depend on either ImageMagick or GraphicsMagick being installed.