Skip to content

Commit

Permalink
Rewrite the testdriver docs.
Browse files Browse the repository at this point in the history 
This extracts most of the documentation from the JSDoc comments rather
than maintaining it in two places.

Due to the limitations of sphinx-js, it seems like every command needs
to be included one-by-one in the docs. However that has the advantage
that we can give them some orgnaisation rather than just providing a
flat list of commands.
  • Loading branch information
@jgraham
jgraham committed Nov 5, 2021
1 parent e45e97f commit 7230491
Show file tree
Hide file tree
Showing 3 changed files with 293 additions and 213 deletions.
271 changes: 121 additions & 150 deletions docs/writing-tests/testdriver.md
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,13 @@
# testdriver.js Automation

```eval_rst
.. contents:: Table of Contents
:depth: 3
:local:
:backlinks: none
```

testdriver.js provides a means to automate tests that cannot be
written purely using web platform APIs. Outside of automation
contexts, it allows human operators to provide expected input
Expand All @@ -8,130 +16,153 @@ manually (for operations which may be described in simple terms).
It is currently supported only for [testharness.js](testharness)
tests.

## API
## Markup ##

The `testdriver.js` and `testdriver-vendor.js` must both be included
in any document that uses testdriver (and in the top-level test
document when using testdriver from a different context):

```html
<script src="/resources/testdriver.js"></script>
<script src="/resources/testdriver-vendor.js"></script>
```

## API ##

testdriver.js exposes its API through the `test_driver` variable in
the global scope.

### Actions
Usage:
### User Interaction ###

```eval_rst
.. js:autofunction:: test_driver.click
.. js:autofunction:: test_driver.send_keys
.. js:autofunction:: test_driver.action_sequence
.. js:autofunction:: test_driver.bless
```
let actions = new test_driver.Actions()
.action1()
.action2();
actions.send()

### Cookies ###
```eval_rst
.. js:autofunction:: test_driver.delete_all_cookies
```

Test authors are encouraged to use the builder API to generate the
sequence of actions. The builder API can be accessed via the `new
test_driver.Actions()` object, and actions are defined in
[testdriver-actions.js](https://github.com/web-platform-tests/wpt/blob/master/resources/testdriver-actions.js)
### Permissions ###
```eval_rst
.. js:autofunction:: test_driver.set_permission
```

The `actions.send()` function causes the sequence of actions to be
sent to the browser. It is based on the [WebDriver
API](https://w3c.github.io/webdriver/#actions). The action can be a
keyboard action, a pointer action or a pause. It returns a promise
that resolves after the actions have been sent, or rejects if an error
was thrown.
### Authentication ###

```eval_rst
.. js:autofunction:: test_driver.add_virtual_authenticator
.. js:autofunction:: test_driver.remove_virtual_authenticator
.. js:autofunction:: test_driver.add_credential
.. js:autofunction:: test_driver.get_credentials
.. js:autofunction:: test_driver.remove_credential
.. js:autofunction:: test_driver.remove_all_credentials
.. js:autofunction:: test_driver.set_user_verified
```

Example:
### Page Lifecycle ###
```eval_rst
.. js:autofunction:: test_driver.freeze
```

```js
let text_box = document.getElementById("text");
### Reporting Observer ###
```eval_rst
.. js:autofunction:: test_driver.generate_test_report
```

let actions = new test_driver.Actions()
.pointerMove(0, 0, {origin: text_box})
.pointerDown()
.pointerUp()
.addTick()
.keyDown("p")
.keyUp("p");
### Storage ###
```eval_rst
.. js:autofunction:: test_driver.set_storage_access
actions.send();
```

Calling into `send()` is going to dispatch the action sequence (via
`test_driver.action_sequence`) and also returns a promise which should
be handled however is appropriate in the test. The other functions in
the `Actions()` object are going to modify the state of the object by
adding a new action in the sequence and returning the same object. So
the functions can be easily chained, as shown in the example
above. Here is a list of helper functions in the `Actions` class:
### Using test_driver in other browsing contexts ###

Testdriver can be used in browsing contexts (i.e. windows or frames)
from which it's possible to get a reference to the top-level test
context. There are two basic approaches depending on whether the
context in which testdriver is used is same-origin with the test
context, or different origin.

For same-origin contexts, the context can be passed directly into the
testdriver API calls. For functions that take an element argument this
is done implicitly using the owner document of the element. For
functions that don't take an element, this is done via an explicit
context argument, which takes a WindowProxy object.

Example:
```
pointerDown: Create a pointerDown event for the current default pointer source
pointerUp: Create a pointerUp event for the current default pointer source
pointerMove: Create a move event for the current default pointer source
keyDown: Create a keyDown event for the current default key source
keyUp: Create a keyUp event for the current default key source
pause: Add a pause to the current tick
addTick: Insert a new actions tick
setPointer: Set the current default pointer source (By detault the pointerType is mouse)
addPointer: Add a new pointer input source with the given name
setKeyboard: Set the current default key source
addKeyboard: Add a new key input source with the given name
let win = window.open("example.html")
win.onload = () => {
await test_driver.set_permission({ name: "background-fetch" }, "denied", win);
}
```

This works with elements in other frames/windows as long as they are
same-origin with the test, and the test does not depend on the
window.name property remaining unset on the target window.
```eval_rst
.. js:autofunction:: test_driver.set_test_context
.. js:autofunction:: test_driver.message_test
```

### bless
For cross-origin cases, passing in the `context` doesn't work because
of limitations in the WebDriver protocol used to implement testdriver
in a cross-browser fashion. Instead one may include the testdriver
scripts directly in the relevant document, and use the
[`test_driver.set_test_context`](#test_driver.set_test_context) API to
specify the browsing context containing testharness.js. Commands are
then sent via `postMessage` to the test context. For convenience there
is also a [`test_driver.message_test`](#test_driver.message_test)
function that can be used to send arbitary messages to the test
window. For example, in an auxillary browsing context:

Usage: `test_driver.bless(intent, action)`
* _intent_: a string describing the motivation for this invocation
* _action_: an optional function
```js
testdriver.set_test_context(window.opener)
await testdriver.click(document.getElementsByTagName("button")[0])
testdriver.message_test("click complete")
```

This function simulates [activation][activation], allowing tests to
perform privileged operations that require user interaction. For
example, sandboxed iframes with
`allow-top-navigation-by-user-activation` may only navigate their
parent's browsing context under these circumstances. The _intent_
string is presented to human operators when the test is not run in
automation.
The requirement to have a handle to the test window does mean it's
currently not possible to write tests where such handles can't be
obtained e.g. in the case of `rel=noopener`.

This method returns a promise which is resolved with the result of
invoking the _action_ function. If no such function is provided, the
promise is resolved with the value `undefined`.
## Actions ##

Example:
### Markup ###

```js
var mediaElement = document.createElement('video');
To use the [Actions](#Actions) API `testdriver-actions.js` must be
included in the document, in addition to `testdriver.js`:

test_driver.bless('initiate media playback', function () {
mediaElement.play();
});
```html
<script src="/resources/testdriver-actions.js"></script>
```

### click

Usage: `test_driver.click(element)`
* _element_: a DOM Element object
### API ###

This function causes a click to occur on the target element (an
`Element` object), potentially scrolling the document to make it
possible to click it. It returns a promise that resolves after the
click has occurred or rejects if the element cannot be clicked (for
example, it is obscured by an element on top of it).
```eval_rst
.. js:autoclass:: Actions
:members:
```

This works with elements in other frames/windows as long as they are
same-origin with the test, and the test does not depend on the
window.name property remaining unset on the target window.

Note that if the element to be clicked does not have a unique ID, the
document must not have any DOM mutations made between the function
being called and the promise settling.
### Using in other browsing contexts ###

## delete_all_cookies
For the actions API, the context can be set using the `setContext`
method on the builder:

Usage: `test_driver.delete_all_cookies(context=null)`
* _context_: an optional WindowProxy for the browsing context in which to
perform the call. Defaults to the current browsing context.
```js
let actions = new test_driver.Actions()
.setContext(frames[0])
.keyDown("p")
.keyUp("p");
actions.send();
```

This function returns a promise that resolves after all the cookies have
been deleted from the provided browsing context.
Note that if an action uses an element reference, the context will be
derived from that element, and must match any explictly set
context. Using elements in multiple contexts in a single action chain
is not supported.

### send_keys

Expand Down Expand Up @@ -184,63 +215,3 @@ Example:
await test_driver.set_permission({ name: "background-fetch" }, "denied");
await test_driver.set_permission({ name: "push", userVisibleOnly: true }, "granted", true);
```

## Using testdriver in Other Browsing Contexts

Testdriver can be used in browsing contexts (i.e. windows or frames)
from which it's possible to get a reference to the top-level test
context. There are two basic approaches depending on whether the
context in which testdriver is used is same-origin with the test
context, or different origin.

For same-origin contexts, the context can be passed directly into the
testdriver API calls. For functions that take an element argument this
is done implicitly using the owner document of the element. For
functions that don't take an element, this is done via an explicit
context argument, which takes a WindowProxy object.

Example:
```
let win = window.open("example.html")
win.onload = () => {
await test_driver.set_permission({ name: "background-fetch" }, "denied", win);
}
```

For the actions API, the context can be set using the `setContext`
method on the builder:

```
let actions = new test_driver.Actions()
.setContext(frames[0])
.keyDown("p")
.keyUp("p");
actions.send();
```

Note that if an action uses an element reference, the context will be
derived from that element, and must match any explictly set
context. Using elements in multiple contexts in a single action chain
is not supported.


For cross-origin cases, passing in the context id doesn't work because
of limitations in the WebDriver protocol used to implement testdriver
in a cross-browser fashion. Instead one may include the testdriver
scripts directly in the relevant document, and use the
`set_test_context` API to specify the browsing context containing
testharness.js. Commands are then sent via postMessage to the test
context. For convenience there is also a `message_test` function that
can be used to send arbitary messages to the test window. For example,
in an auxillary browsing context:


```
testdriver.set_test_context(window.opener)
await testdriver.click(document.getElementsByTagName("button")[0])
testdriver.message_test("click complete")
```

The requirement to have a handle to the test window does mean it's
currently not possible to write tests where such handles can't be
obtained e.g. in the case of `rel=noopener`.
33 changes: 31 additions & 2 deletions resources/testdriver-actions.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,38 @@
let sourceNameIdx = 0;

/**
* @class
* Builder for creating a sequence of actions
* The default tick duration is set to 16ms, which is one frame time based on
* 60Hz display.
*
*
* The actions are dispatched once
* :js:func:`test_driver.Actions.send` is called. This returns a
* promise which resolves once the actions are complete.
*
* The other methods on :js:class:`test_driver.Actions` object are
* used to build the sequence of actions that will be sent. These
* return the `Actions` object itself, so the actions sequence can
* be constructed by chaining method calls.
*
* Internally :js:func:`test_driver.Actions.send` invokes
* :js:func:`test_driver.action_sequence`.
*
* @example
* let text_box = document.getElementById("text");
*
* let actions = new test_driver.Actions()
* .pointerMove(0, 0, {origin: text_box})
* .pointerDown()
* .pointerUp()
* .addTick()
* .keyDown("p")
* .keyUp("p");
*
* actions.send();
*
* @param {number} [defaultTickDuration] - The default duration of a
* tick. Be default this is set ot 16ms, which is one frame time
* based on 60Hz display.
*/
function Actions(defaultTickDuration=16) {
this.sourceTypes = new Map([["key", KeySource],
Expand Down
Loading

0 comments on commit 7230491

Please sign in to comment.