docs: update docs for v3 (GoogleChrome#5357)

Author: patrickhulce

CONTRIBUTING.md

@@ -4,7 +4,7 @@ We'd love your help! This doc covers how to become a contributor and submit code
 
 ## Follow the coding style
 
-The `.eslintrc` file defines all. We use [JSDoc](http://usejsdoc.org/) along with [closure annotations](https://github.com/google/closure-compiler/wiki/Annotating-JavaScript-for-the-Closure-Compiler). Annotations are encouraged for all contributions.
+The `.eslintrc` file defines all. We use [JSDoc](http://usejsdoc.org/) with [TypeScript](https://github.com/Microsoft/TypeScript/wiki/JSDoc-support-in-JavaScript). Annotations are encouraged for all contributions.
 
 ## Pull request titles
 

docs/architecture.md

@@ -84,6 +84,6 @@ Tracing processor takes the output of trace of tab and identifies the top-level
 
 ## Audits
 
-The return value of each audit [takes this shape](https://github.com/GoogleChrome/lighthouse/blob/b354890076f2c077c5460b2fa56ded546cca72ee/lighthouse-core/closure/typedefs/AuditResult.js#L23-L55).
+The return value of each audit [takes this shape](https://github.com/GoogleChrome/lighthouse/blob/8f500e00243e07ef0a80b39334bedcc8ddc8d3d0/typings/audit.d.ts#L117-L130).
 
 The `details` object is parsed in report-renderer.js. View other audits for guidance on how to structure `details`.

docs/configuration.md

@@ -11,20 +11,16 @@ You can specify a custom config file when using Lighthouse through the CLI or co
 **custom-config.js**
 ```js
 module.exports = {
-  passes: [{
-    recordTrace: true,
-    pauseAfterLoadMs: 5000,
-    useThrottling: true,
-    gatherers: [],
-  }],
-
-  audits: [
-    'first-meaningful-paint',
-    'speed-index-metric',
-    'estimated-input-latency',
-    'first-interactive',
-    'consistently-interactive',
-  ]
+  extends: 'lighthouse:default',
+  settings: {
+    onlyAudits: [
+      'first-meaningful-paint',
+      'speed-index-metric',
+      'estimated-input-latency',
+      'first-interactive',
+      'consistently-interactive',
+    ],
+  },
 };
 ```
 
@@ -77,6 +73,8 @@ The settings property controls various aspects of running Lighthouse such as CPU
 ```
 
 #### Options
+For full list see [our default config settings](https://github.com/GoogleChrome/lighthouse/blob/8f500e00243e07ef0a80b39334bedcc8ddc8d3d0/lighthouse-core/config/constants.js#L30-L48).
+
 | Name | Type | Description |
 | -- | -- | -- |
 | onlyCategories | `string[]` | Includes only the specified categories in the final report. Additive with `onlyAudits` and reduces the time to audit a page. |
@@ -89,22 +87,21 @@ The passes property controls how to load the requested URL and what information
 
 Each `passes` entry defines basic settings such as how long to wait for the page to load and whether to record a trace file. Additionally a list of **gatherers** to use is defined per pass. Gatherers can read information from the page to generate artifacts which are later used by audits to provide you with a Lighthouse report. For more information on implementing a custom gatherer and the role they play in building a Lighthouse report, refer to the [recipes](https://github.com/GoogleChrome/lighthouse/blob/master/docs/recipes/custom-audit). Also note that `artifacts.devtoolsLogs` will be automatically populated for every pass. Gatherers also have access to this data within the `afterPass` as `traceData.devtoolsLog` (However, most will find the higher-level `traceData.networkRecords` more useful).
 
+For list of default pass values, see [our config constants](https://github.com/GoogleChrome/lighthouse/blob/8f500e00243e07ef0a80b39334bedcc8ddc8d3d0/lighthouse-core/config/constants.js#L50-L61).
 
 #### Example
 ```js
 {
   passes: [
     {
       passName: 'fastPass',
-      recordTrace: true,
-      useThrottling: false,
-      networkQuietThresholdMs: 0,
       gatherers: ['fast-gatherer'],
     },
     {
       passName: 'slowPass',
       recordTrace: true,
       useThrottling: true,
+      networkQuietThresholdMs: 5000,
       gatherers: ['slow-gatherer'],
     }
   ]
@@ -141,7 +138,7 @@ The audits property controls which audits to run and include with your Lighthous
 
 ### `categories: Object|undefined`
 
-The categories property controls how to score and organize the audit results in the report. Each category defined in the config will have an entry in the `reportCategories` property of Lighthouse's output. The category output contains the child audit results along with an overall score for the category.
+The categories property controls how to score and organize the audit results in the report. Each category defined in the config will have an entry in the `categories` property of Lighthouse's output. The category output contains the child audit results along with an overall score for the category.
 
 **Note:** many modules consuming Lighthouse have no need to group or score all the audit results; in these cases, it's fine to omit a categories section.
 
@@ -150,9 +147,9 @@ The categories property controls how to score and organize the audit results in
 {
   categories: {
     performance: {
-      name: 'Performance',
+      title: 'Performance',
       description: 'This category judges your performance',
-      audits: [
+      auditRefs: [
         {id: 'first-meaningful-paint', weight: 2, group: 'metrics'},
         {id: 'first-interactive', weight: 3, group: 'metrics'},
         {id: 'consistently-interactive', weight: 5, group: 'metrics'},
@@ -165,12 +162,12 @@ The categories property controls how to score and organize the audit results in
 #### Options
 | Name | Type | Description |
 | -- | -- | -- |
-| name | `string` | The display name of the category. |
+| title | `string` | The display name of the category. |
 | description | `string` | The displayed description of the category. |
-| audits | `Object[]` | The audits to include in the category. |
-| audits[$i].id | `string` | The ID of the audit to include. |
-| audits[$i].weight | `number` | The weight of the audit in the scoring of the category. |
-| audits[$i].group | `string` (optional) | The ID of the [display group](#groups-objectundefined) of the audit. |
+| auditRefs | `Object[]` | The audits to include in the category. |
+| auditRefs[$i].id | `string` | The ID of the audit to include. |
+| auditRefs[$i].weight | `number` | The weight of the audit in the scoring of the category. |
+| auditRefs[$i].group | `string` (optional) | The ID of the [display group](#groups-objectundefined) of the audit. |
 
 ### `groups: Object|undefined`
 
@@ -183,7 +180,7 @@ The groups property controls how to visually group audits within a category. For
 {
   categories: {
     performance: {
-      audits: [
+      auditRefs: [
         {id: 'my-performance-metric', weight: 2, group: 'metrics'},
       ],
     }
@@ -208,7 +205,7 @@ The stock Lighthouse configurations can be extended if you only need to make sma
 The best examples are the ones Lighthouse uses itself! There are several reference configuration files that are maintained as part of Lighthouse.
 
 * [lighthouse-core/config/default-config.js](https://github.com/GoogleChrome/lighthouse/blob/master/lighthouse-core/config/default-config.js)
-* [lighthouse-core/config/plots-config.js](https://github.com/GoogleChrome/lighthouse/blob/master/lighthouse-core/config/plots-config.js)
+* [lighthouse-core/config/perf-config.js](https://github.com/GoogleChrome/lighthouse/blob/master/lighthouse-core/config/perf-config.js)
 * [docs/recipes/custom-audit/custom-config.js](https://github.com/GoogleChrome/lighthouse/blob/master/docs/recipes/custom-audit/custom-config.js)
 * [pwmetrics](https://github.com/paulirish/pwmetrics/blob/master/lib/lh-config.ts)
 

docs/headless-chrome.md

@@ -27,8 +27,8 @@ lighthouse --chrome-flags="--headless" https://github.com
 Alternatively, you can run full Chrome + xvfb instead of headless mode. These steps worked on Debian Jessie:
 
 ```sh
-# get node 6
-curl -sL https://deb.nodesource.com/setup_6.x | sudo -E bash -
+# get node 8
+curl -sL https://deb.nodesource.com/setup_8.x | sudo -E bash -
 sudo apt-get install -y nodejs
 
 # get chromium (stable) and Xvfb

docs/lantern.md

@@ -29,7 +29,7 @@ Stats were collected using the [trace-evaluation](https://github.com/patrickhulc
 We conclude that Lantern is ~6-13% more inaccurate than DevTools throttling. When evaluating rank performance, Lantern achieves correlations within ~.04-.07 of DevTools throttling.
 
 * For the single view use case, our original conclusion that Lantern's inaccuracy is roughly equal to the inaccuracy introduced by expected variance seems to hold. The standard deviation of single observations from DevTools throttling is ~9-13%, and given Lantern's much lower variance, single observations from Lantern are not significantly more inaccurate on average than single observations from DevTools throttling.
-* For the repeat view use case, we can conclude that Lantern is systematically off by ~6-13% more than DevTools throttling. 
+* For the repeat view use case, we can conclude that Lantern is systematically off by ~6-13% more than DevTools throttling.
 
 ### Metric Variability Conclusions
 The reference stats demonstrate that there is high degree of variability with the user-centric metrics and strengthens the position that every load is just an observation of a point drawn from a distribution and to understand the entire experience, multiple draws must be taken, i.e. multiple runs are needed to have sufficiently small error bounds on the median load experience.

docs/puppeteer.md

@@ -45,13 +45,13 @@ browser.on('targetchanged', async target => {
 
 // Lighthouse will open URL. Puppeteer observes `targetchanged` and sets up network conditions.
 // Possible race condition.
-const lhr = await lighthouse(url, {
+const {lhr} = await lighthouse(url, {
   port: (new URL(browser.wsEndpoint())).port,
   output: 'json',
   logLevel: 'info',
 });
 
-console.log(`Lighthouse scores: ${lhr.reportCategories.map(c => c.score).join(', ')}`);
+console.log(`Lighthouse scores: ${Object.values(lhr.categories).map(c => c.score).join(', ')}`);
 
 await browser.close();
 })();
@@ -89,8 +89,8 @@ const {webSocketDebuggerUrl} = JSON.parse(resp.body);
 const browser = await puppeteer.connect({browserWSEndpoint: webSocketDebuggerUrl});
 
 // Run Lighthouse.
-const lhr = await lighthouse(URL, opts, null);
-console.log(`Lighthouse scores: ${lhr.reportCategories.map(c => c.score).join(', ')}`);
+const {lhr}  = await lighthouse(URL, opts, null);
+console.log(`Lighthouse scores: ${Object.values(lhr.categories).map(c => c.score).join(', ')}`);
 
 await browser.disconnect();
 await chrome.kill();

docs/readme.md

@@ -15,9 +15,11 @@ function launchChromeAndRunLighthouse(url, opts, config = null) {
   return chromeLauncher.launch({chromeFlags: opts.chromeFlags}).then(chrome => {
     opts.port = chrome.port;
     return lighthouse(url, opts, config).then(results => {
-      // The gathered artifacts are typically removed as they can be quite large (~50MB+)
-      delete results.artifacts;
-      return chrome.kill().then(() => results)
+      // use results.lhr for the JS-consumeable output
+      // https://github.com/GoogleChrome/lighthouse/blob/master/typings/lhr.d.ts
+      // use results.report for the HTML/JSON/CSV output as a string
+      // use results.artifacts for the trace/screenshots/other specific case you need (rarer)
+      return chrome.kill().then(() => results.lhr)
     });
   });
 }
@@ -39,13 +41,31 @@ Many modules consuming Lighthouse are only interested in the performance numbers
 You can limit the audits you run to a particular category or set of audits.
 
 ```js
-const perfConfig = require('lighthouse/lighthouse-core/config/perf-config.js');
 // ...
-launchChromeAndRunLighthouse(url, flags, perfConfig).then( // ...
+const flags = {onlyCategories: ['performance']};
+launchChromeAndRunLighthouse(url, flags).then( // ...
 ```
 
 You can also craft your own config (e.g. [mixed-content-config.js](https://github.com/GoogleChrome/lighthouse/blob/master/lighthouse-core/config/mixed-content-config.js)) for custom runs. Also see the [basic custom audit recipe](https://github.com/GoogleChrome/lighthouse/tree/master/docs/recipes/custom-audit).
 
+### Differences from CLI flags
+
+Note that some flag functionality is only available to the CLI. The set of shared flags that work in both node and CLI can be found [in our typedefs](https://github.com/GoogleChrome/lighthouse/blob/8f500e00243e07ef0a80b39334bedcc8ddc8d3d0/typings/externs.d.ts#L68). In most cases, the functionality is not offered in the node module simply because it is easier and more flexible to do it yourself.
+
+| CLI Flag | Differences in Node |
+| - | - |
+| `port` | Only specifies which port to use, Chrome is not launched for you. |
+| `chromeFlags` | Ignored, Chrome is not launched for you. |
+| `outputPath` | Ignored, output is returned as string in `.report` property. |
+| `saveAssets` | Ignored, artifacts are returned in `.artifacts` property. |
+| `view` | Ignored, use the `opn` npm module if you want this functionality. |
+| `enableErrorReporting` | Ignored, error reporting is always disabled for node. |
+| `listAllAudits` | Ignored, not relevant in programmatic use. |
+| `listTraceCategories` | Ignored, not relevant in programmatic use. |
+| `configPath` | Ignored, pass the config in as the 3rd argument to `lighthouse`. |
+| `preset` | Ignored, pass the config in as the 3rd argument to `lighthouse`. |
+| `verbose` | Ignored, use `logLevel` instead. |
+| `quiet` | Ignored, use `logLevel` instead. |
 
 ### Turn on logging
 
@@ -56,7 +76,7 @@ the `logLevel` flag when calling `lighthouse`.
 ```javascript
 const log = require('lighthouse-logger');
 
-const flags = {logLevel: 'info', output: 'json'};
+const flags = {logLevel: 'info'};
 log.setLevel(flags.logLevel);
 
 launchChromeAndRunLighthouse('https://example.com', flags).then(...);

docs/scoring.md

@@ -6,7 +6,7 @@ Note 1: if you want a **nice spreadsheet** version of this doc to understand wei
 ![alt text](https://user-images.githubusercontent.com/39191/32397461-2d20c87a-c0a7-11e7-99d8-61576113a710.png)
 *Screenshot of the [scoring spreadsheet](https://docs.google.com/spreadsheets/d/1Cxzhy5ecqJCucdf1M0iOzM8mIxNc7mmx107o5nj38Eo/edit#gid=0)*
 
-Note 2: if you receive a **score of 0** in any Lighthouse category, that usually indicates an error on our part. Please file an [issue](https://github.com/GoogleChrome/lighthouse/issues) so our team can look into it.
+Note 2: receiving a **score of ?** in any Lighthouse category indicates an error occurred. Please file an [issue](https://github.com/GoogleChrome/lighthouse/issues) so our team can look into it.
 
 # Performance
 
@@ -30,11 +30,12 @@ The performance score is determined from the **performance metrics only**. The O
 
 The metric results are not weighted equally. Currently the weights are:
 
-* 5X - first meaningful paint
-* 5X - first interactive
+* 3X - first contentful paint
+* 1X - first meaningful paint
+* 3X - first cpu idle
 * 5X - consistently interactive
-* 1X - perceptual speed index
-* 1X - estimated input latency
+* 4X - speed index
+* 0X - estimated input latency
 
 These weights are heuristics, and the Lighthouse team is working on formalizing the weighting system through more field data.
 
@@ -47,7 +48,7 @@ Once we finish computing the percentile equivalent of your raw performance score
 - Green (good): 75-100.
 
 ### What can developers do to improve their performance score?
-*Note: we've built [a little calculator](https://docs.google.com/spreadsheets/d/1dXH-bXX3gxqqpD1f7rp6ImSOhobsT1gn_GQ2fGZp8UU/edit?ts=59fb61d2#gid=283330180) that can help you understand what thresholds you should be aiming for achieving a certain Lighthouse performance score. *
+*Note: we've built [a little calculator](https://docs.google.com/spreadsheets/d/1Cxzhy5ecqJCucdf1M0iOzM8mIxNc7mmx107o5nj38Eo/edit#gid=283330180) that can help you understand what thresholds you should be aiming for achieving a certain Lighthouse performance score. *
 
 Lighthouse has a whole section in the report on improving your performance score under the “Opportunities” section. There are detailed suggestions and documentation that explains the different suggestions and how to implement them. Additionally, the diagnostics section lists additional guidance that developers can explore to further experiment and tweak with their performance.
 

docs/throttling.md

@@ -7,11 +7,12 @@ This is the standard recommendation for 3G throttling:
 - Throughput: 1.6Mbps down / 750 Kbps up.
 - Packet loss: none.
 
-These exact figures are used as the [WebPageTest "Mobile 3G - Fast" preset](https://github.com/WPO-Foundation/webpagetest/blob/master/www/settings/connectivity.ini.sample) and [Lighthouse's throttling default](https://github.com/GoogleChrome/lighthouse/blob/master/lighthouse-core/lib/emulation.js).
+These exact figures are used as the [WebPageTest "Mobile 3G - Fast" preset](https://github.com/WPO-Foundation/webpagetest/blob/master/www/settings/connectivity.ini.sample) and [Lighthouse's throttling default](https://github.com/GoogleChrome/lighthouse/blob/8f500e00243e07ef0a80b39334bedcc8ddc8d3d0/lighthouse-core/config/constants.js#L19-L26).
 
 ## Throttling basics
 
-1. The DevTools network throttling, which Lighthouse uses, is implemented within Chrome at the _request-level_. As a result, it has some downsides that are now summarized in this doc: [Network Throttling & Chrome - status](https://docs.google.com/document/d/1TwWLaLAfnBfbk5_ZzpGXegPapCIfyzT4MWuZgspKUAQ/edit). The TLDR: while it's a [decent approximation](https://docs.google.com/document/d/1uS9SH1KpVH31JAmf-iIZ61VazwAF9MrCVwETshBC4UQ/edit), it's not a sufficient model of a slow connection. The [multipliers used in Lighthouse](https://github.com/GoogleChrome/lighthouse/blob/3be483287a530fb560c843b7299ef9cfe91ce1cc/lighthouse-core/lib/emulation.js#L33-L39) attempt to correct for the differences.
+1. Simulated throttling, which Lighthouse uses by default, is computed throttling _after a trace has been recorded_ which makes it very fast and deterministic. However, due to the imperfect nature of predicting alternate execution paths, there is inherent inaccuracy that is summarized in this doc: [Lighthouse Metric Variability and Accuracy](https://docs.google.com/document/d/1BqtL-nG53rxWOI5RO0pItSRPowZVnYJ_gBEQCJ5EeUE/edit). The TLDR: while it's roughly as accurate or better than DevTools throttling for most sites, it suffers from edge cases and a deep investigation to performance should use _Packet-level_ throttling tools.
+1. The DevTools network throttling, which Lighthouse uses for `throttlingMethod='devtools'`, is implemented within Chrome at the _request-level_. As a result, it has some downsides that are now summarized in this doc: [Network Throttling & Chrome - status](https://docs.google.com/document/d/1TwWLaLAfnBfbk5_ZzpGXegPapCIfyzT4MWuZgspKUAQ/edit). The TLDR: while it's a [decent approximation](https://docs.google.com/document/d/1uS9SH1KpVH31JAmf-iIZ61VazwAF9MrCVwETshBC4UQ/edit), it's not a sufficient model of a slow connection. The [multipliers used in Lighthouse](https://github.com/GoogleChrome/lighthouse/blob/3be483287a530fb560c843b7299ef9cfe91ce1cc/lighthouse-core/lib/emulation.js#L33-L39) attempt to correct for the differences.
 1. _Proxy-level_ throttling tools do not affect UDP data, so they're not recommended.
 1. _Packet-level_ throttling tools are able to make the most accurate network simulation.