Skip to content

Commit

Permalink
Mouse (puppeteer#101)
Browse files Browse the repository at this point in the history
This patch:
- adds Mouse class which holds mouse state and implements mouse primitives,
such as moving, button down and button up.
- implements high-level mouse api, such as `page.click` and `page.hover`.

References puppeteer#40, References puppeteer#89
  • Loading branch information
JoelEinbinder authored and aslushnikov committed Jul 22, 2017
1 parent eb2cb67 commit 98ee356
Show file tree
Hide file tree
Showing 17 changed files with 458 additions and 55 deletions.
69 changes: 67 additions & 2 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,18 +29,20 @@
+ [page.$(selector, pageFunction, ...args)](#pageselector-pagefunction-args)
+ [page.$$(selector, pageFunction, ...args)](#pageselector-pagefunction-args)
+ [page.addScriptTag(url)](#pageaddscripttagurl)
+ [page.click(selector)](#pageclickselector)
+ [page.click(selector[, options])](#pageclickselector-options)
+ [page.close()](#pageclose)
+ [page.evaluate(pageFunction, ...args)](#pageevaluatepagefunction-args)
+ [page.evaluateOnInitialized(pageFunction, ...args)](#pageevaluateoninitializedpagefunction-args)
+ [page.focus(selector)](#pagefocusselector)
+ [page.frames()](#pageframes)
+ [page.goBack(options)](#pagegobackoptions)
+ [page.goForward(options)](#pagegoforwardoptions)
+ [page.hover(selector)](#pagehoverselector)
+ [page.httpHeaders()](#pagehttpheaders)
+ [page.injectFile(filePath)](#pageinjectfilefilepath)
+ [page.keyboard](#pagekeyboard)
+ [page.mainFrame()](#pagemainframe)
+ [page.mouse](#pagemouse)
+ [page.navigate(url, options)](#pagenavigateurl-options)
+ [page.pdf(options)](#pagepdfoptions)
+ [page.plainText()](#pageplaintext)
Expand All @@ -67,6 +69,11 @@
+ [keyboard.modifiers()](#keyboardmodifiers)
+ [keyboard.sendCharacter(char)](#keyboardsendcharacterchar)
+ [keyboard.up(key)](#keyboardupkey)
* [class: Mouse](#class-mouse)
+ [mouse.down([options])](#mousedownoptions)
+ [mouse.move(x, y)](#mousemovex-y)
+ [mouse.press([options])](#mousepressoptions)
+ [mouse.up([options])](#mouseupoptions)
* [class: Dialog](#class-dialog)
+ [dialog.accept([promptText])](#dialogacceptprompttext)
+ [dialog.dismiss()](#dialogdismiss)
Expand All @@ -76,7 +83,9 @@
+ [frame.$(selector, pageFunction, ...args)](#frameselector-pagefunction-args)
+ [frame.$$(selector, pageFunction, ...args)](#frameselector-pagefunction-args)
+ [frame.childFrames()](#framechildframes)
+ [frame.click(selector[, options])](#frameclickselector-options)
+ [frame.evaluate(pageFunction, ...args)](#frameevaluatepagefunction-args)
+ [frame.hover(selector)](#framehoverselector)
+ [frame.isDetached()](#frameisdetached)
+ [frame.isMainFrame()](#frameismainframe)
+ [frame.name()](#framename)
Expand Down Expand Up @@ -351,8 +360,11 @@ Shortcut for [page.mainFrame().$$(selector, pageFunction, ...args)](#pageselecto

Adds a `<script></script>` tag to the page with the desired url. Alternatively, javascript could be injected to the page via `page.injectFile` method.

#### page.click(selector)
#### page.click(selector[, options])
- `selector` <[string]> A query selector to search for element to click. If there are multiple elements satisfying the selector, the first will be clicked.
- `options` <[Object]>
- `button` <[string]> `left`, `right`, or `middle`, defaults to `left`.
- `clickCount` <[number]> defaults to 1
- returns: <[Promise]> Promise which resolves when the element matching `selector` is successfully clicked. Promise gets rejected if there's no element matching `selector`.

#### page.close()
Expand Down Expand Up @@ -393,6 +405,10 @@ can not go back, resolves to null.

Navigate to the next page in history.

#### page.hover(selector)
- `selector` <[string]> A query selector to search for element to hover. If there are multiple elements satisfying the selector, the first will be hovered.
- returns: <[Promise]> Promise which resolves when the element matching `selector` is successfully hovered. Promise gets rejected if there's no element matching `selector`.

#### page.httpHeaders()
- returns: <[Object]> Key-value set of additional http headers which will be sent with every request.

Expand All @@ -409,6 +425,10 @@ Navigate to the next page in history.

Page is guaranteed to have a main frame which persists during navigations.

#### page.mouse

- returns: <[Mouse]>

#### page.navigate(url, options)
- `url` <[string]> URL to navigate page to
- `options` <[Object]> Navigation parameters which might have the following properties:
Expand Down Expand Up @@ -671,6 +691,39 @@ page.keyboard.sendCharacter('嗨');

Dispatches a `keyup` event.

### class: Mouse

#### mouse.down([options])
- `options` <[Object]>
- `button` <[string]> `left`, `right`, or `middle`, defaults to `left`.
- `clickCount` <[number]> defaults to 1
- returns: <[Promise]>

Dispatches a `mousedown` event.

#### mouse.move(x, y)
- `x` <[number]>
- `y` <[number]>
- returns: <[Promise]>

Dispatches a `mousemove` event.

#### mouse.press([options])
- `options` <[Object]>
- `button` <[string]> `left`, `right`, or `middle`, defaults to `left`.
- `clickCount` <[number]> defaults to 1
- returns: <[Promise]>

Shortcut for [`mouse.down`](#mousedownkey) and [`mouse.up`](#mouseupkey).

#### mouse.up([options])
- `options` <[Object]>
- `button` <[string]> `left`, `right`, or `middle`, defaults to `left`.
- `clickCount` <[number]> defaults to 1
- returns: <[Promise]>

Dispatches a `mouseup` event.

### class: Dialog

[Dialog] objects are dispatched by page via the ['dialog'](#event-dialog) event.
Expand Down Expand Up @@ -749,6 +802,13 @@ browser.newPage().then(async page => {
#### frame.childFrames()
- returns: <[Array]<[Frame]>>

#### frame.click(selector[, options])
- `selector` <[string]> A query selector to search for element to click. If there are multiple elements satisfying the selector, the first will be clicked.
- `options` <[Object]>
- `button` <[string]> `left`, `right`, or `middle`, defaults to `left`.
- `clickCount` <[number]> defaults to 1
- returns: <[Promise]> Promise which resolves when the element matching `selector` is successfully clicked. Promise gets rejected if there's no element matching `selector`.

#### frame.evaluate(pageFunction, ...args)
- `pageFunction` <[function]> Function to be evaluated in browser context
- `...args` <...[string]> Arguments to pass to `pageFunction`
Expand All @@ -768,6 +828,10 @@ browser.newPage().then(async page =>
});
```

#### frame.hover(selector)
- `selector` <[string]> A query selector to search for element to hover. If there are multiple elements satisfying the selector, the first will be hovered.
- returns: <[Promise]> Promise which resolves when the element matching `selector` is successfully hovered. Promise gets rejected if there's no element matching `selector`.

#### frame.isDetached()
- returns: <[boolean]>

Expand Down Expand Up @@ -1011,3 +1075,4 @@ If there's already a header with name `name`, the header gets overwritten.
[Element]: https://developer.mozilla.org/en-US/docs/Web/API/element "Element"
[Keyboard]: https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#class-keyboard "Keyboard"
[Dialog]: https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#class-dialog "Dialog"
[Mouse]: https://github.com/GoogleChrome/puppeteer/blob/master/docs/api.md#class-mouse "Mouse"
42 changes: 39 additions & 3 deletions lib/FrameManager.js
Original file line number Diff line number Diff line change
Expand Up @@ -20,20 +20,23 @@ let helper = require('./helper');
class FrameManager extends EventEmitter {
/**
* @param {!Connection} client
* @param {!Mouse} mouse
* @return {!Promise<!FrameManager>}
*/
static async create(client) {
static async create(client, mouse) {
let mainFramePayload = await client.send('Page.getResourceTree');
return new FrameManager(client, mainFramePayload.frameTree);
return new FrameManager(client, mainFramePayload.frameTree, mouse);
}

/**
* @param {!Connection} client
* @param {!Object} frameTree
* @param {!Mouse} mouse
*/
constructor(client, frameTree) {
constructor(client, frameTree, mouse) {
super();
this._client = client;
this._mouse = mouse;
/** @type {!Map<string, !Frame>} */
this._frames = new Map();
this._mainFrame = this._addFramesRecursively(null, frameTree);
Expand Down Expand Up @@ -405,6 +408,39 @@ class Frame {
return this._frameManager._evaluateOnFrame(this, expression);
}

/**
* @param {string} selector
* @return {!Promise}
*/
async hover(selector) {
let center = await this.evaluate(selector => {
let element = document.querySelector(selector);
if (!element)
return null;
element.scrollIntoViewIfNeeded();
let rect = element.getBoundingClientRect();
return {
x: (rect.left + rect.right) / 2,
y: (rect.top + rect.bottom) / 2
};
}, selector);
if (!center)
throw new Error('No node found for selector: ' + selector);
await this._frameManager._mouse.move(center.x, center.y);
}

/**
* @param {string} selector
* @param {!Object=} options
* @return {!Promise}
*/
async click(selector, options) {
await this.hover(selector);
await this._frameManager._mouse.press(options);
// This is a hack for now, to make clicking less race-prone
await this.evaluate(() => new Promise(f => requestAnimationFrame(f)));
}

/**
* @param {?Object} framePayload
*/
Expand Down
106 changes: 106 additions & 0 deletions lib/Mouse.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
/**
* Copyright 2017 Google Inc. All rights reserved.
*
* Licensed under the Apache License, Version 2.0 (the 'License');
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an 'AS IS' BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/

class Mouse {
/**
* @param {!Connection} client
* @param {!Keyboard} keyboard
*/
constructor(client, keyboard) {
this._client = client;
this._keyboard = keyboard;
this._x = 0;
this._y = 0;
this._button = 'none';
}

/**
* @param {number} x
* @param {number} y
* @return {!Promise}
*/
async move(x, y) {
this._x = x;
this._y = y;
await this._client.send('Input.dispatchMouseEvent', {
type: 'mouseMoved',
button: this._button,
x, y,
modifiers: this._modifiersMask()
});
}

/**
* @param {!Object=} options
*/
async press(options) {
await this.down(options);
await this.up(options);
}

/**
* @param {!Object=} options
*/
async down(options) {
if (!options)
options = {};
this._button = (options.button || 'left');
await this._client.send('Input.dispatchMouseEvent', {
type: 'mousePressed',
button: this._button,
x: this._x,
y: this._y,
modifiers: this._modifiersMask(),
clickCount: (options.clickCount || 1)
});
}

/**
* @param {!Object=} options
*/
async up(options) {
if (!options)
options = {};
this._button = 'none';
await this._client.send('Input.dispatchMouseEvent', {
type: 'mouseReleased',
button: (options.button || 'left'),
x: this._x,
y: this._y,
modifiers: this._modifiersMask(),
clickCount: (options.clickCount || 1)
});
}

/**
* @return {number}
*/
_modifiersMask() {
let modifiers = this._keyboard.modifiers();
let mask = 0;
if (modifiers.Alt)
mask += 1;
if (modifiers.Control)
mask += 2;
if (modifiers.Meta)
mask += 4;
if (modifiers.Shift)
mask += 8;
return mask;
}
}

module.exports = Mouse;
Loading

0 comments on commit 98ee356

Please sign in to comment.