Prettier Plugin API (prettier#3536)

* Move files around in preparation for refactor

* Update paths in build script

* Extract generic printing logic from the JavaScript printer

* Conform printer API

* Fixup decorator handling

* Fix multiparser

* Create plugin entry for markdown

* Create plugin entry for javascript/typescript

* Create plugin entry for html

* Create plugin entry for graphql

* Create plugin entry for css/less/scss

* Move JSON to JS plugin entry

* Integrate plugins into getSupportInfo()

* Move astFormat to parser definition

* Move util to common

* Implement parser loading

* remark -> mdast

* Rename cli/cli -> cli/index

* Rename builder -> doc package, fix printer resolution

* Fix doc shape assumption in CSS-in-JS logic

* Fix third-party.js prod resolution

* Fixup build-docs script

* Distribute multiparser code

* Remove requirement to forward options

* Flatten closure

* Remove debug directory

* Expose doc

* Add external plugins

* Pass options to loadPlugins

* Export getParsers

* Pin resolve version

* Use getSupportInfo in Markdown embed

* Document plugin API

* Update build-docs

* Add CLI for plugins

* Lint docs

* Fixup build.js

* Add vue language

* Fixup multiparser for vue

* Upgrade rollup and rollup-plugin-commonjs

* Fixup third-party build

* Change AST format in docs

docs/plugins.md

@@ -0,0 +1,155 @@
+---
+id: plugins
+title: Plugins
+---
+
+# IN DEVELOPMENT
+
+> The plugin API is unreleased and the API may change!
+
+Plugins are ways of adding new languages to Prettier. Prettier's own implementations of all languages are expressed using the plugin API. The core `prettier` package contains JavaScript and other web-focussed languages built in. For additional languages you'll need to install a plugin.
+
+## Using Plugins
+
+There are three ways to add plugins to Prettier:
+
+* Via the CLI.
+* Via the API.
+* With a configuration file.
+
+### Configuration File (Recommended)
+
+In your [configuration file](./configuration.md), add the `plugins` property:
+
+```json
+{
+  "plugins": ["prettier-python"]
+}
+```
+
+### CLI
+
+With the [CLI](./cli.md), pass the `--plugin` flag:
+
+```bash
+prettier --write main.py --plugin prettier-python
+```
+
+> Tip: You can pass multiple `--plugin` flags.
+
+## Official Plugins
+
+* [`prettier-python`](https://github.com/prettier/prettier-python)
+* [`prettier-php`](https://github.com/prettier/prettier-php)
+
+## Developing Plugins
+
+Prettier plugins are regular JavaScript modules with three exports, `languages`, `parsers` and `printers`.
+
+### `languages`
+
+Languages is an array of language definitions that your plugin will contribute to Prettier. It can include all of the fields specified in [`prettier.getSupportInfo()`](./api.md#prettiergetsupportinfo-version).
+
+It **must** include `name` and `parsers`.
+
+```js
+export const languages = [
+  {
+    // The language name
+    name: "InterpretedDanceScript",
+    // Parsers that can parse this language.
+    // This can be built-in parsers, or parsers you have contributed via this plugin.
+    parsers: ["dance-parse"]
+  }
+];
+```
+
+### `parsers`
+
+Parsers convert code as a string into an [AST](https://en.wikipedia.org/wiki/Abstract_syntax_tree).
+
+The key must match the name in the `parsers` array from `languages`. The value contains a parse function and an AST format name.
+
+```js
+export const parsers = {
+  "dance-parse": {
+    parse,
+    // The name of the AST that
+    astFormat: "dance-ast"
+  }
+};
+```
+
+The signature of the `parse` function is:
+
+```ts
+function parse(text: string, parsers: object, options: object): AST;
+```
+
+### `printers`
+
+Printers convert ASTs into a Prettier intermediate representation, also known as a Doc.
+
+The key must match the `astFormat` that the parser produces. The value contains an object with a `print` function and (optionally) an `embed` function.
+
+```js
+export const printers = {
+  "dance-ast": {
+    print,
+    embed
+  }
+};
+```
+
+Printing is a recursive process of coverting an AST node (represented by a path to that node) into a doc. The doc is constructed using the [builder commands](https://github.com/prettier/prettier/blob/master/commands.md):
+
+```js
+const { concat, join, line, ifBreak, group } = require("prettier").doc.builders;
+```
+
+The signature of the `print` function is:
+
+```ts
+function print(
+  // Path to the AST node to print
+  path: FastPath,
+  options: object,
+  // Recursively print a child node
+  print: (path: FastPath) => Doc
+): Doc;
+```
+
+Check out [prettier-python's printer](https://github.com/prettier/prettier-python/blob/034ba8a9551f3fa22cead41b323be0b28d06d13b/src/printer.js#L174) as an example.
+
+Embedding refers to printing one language inside another. Examples of this are CSS-in-JS and Markdown code blocks. Plugins can switch to alternate languages using the `embed` function. Its signature is:
+
+```ts
+function embed(
+  // Path to the current AST node
+  path: FastPath,
+  // Print a node with the current printer
+  print: (path: FastPath) => Doc,
+  // Parse and print some text using a different parser.
+  // You should set `options.parser` to specify which parser to use.
+  textToDoc: (text: string, options: object) => Doc,
+  // Current options
+  options: object
+): Doc | null;
+```
+
+If you don't want to switch to a different parser, simply return `null` or `undefined`.
+
+## Testing Plugins
+
+Since plugins can be resolved using relative paths, when working on one you can do:
+
+```js
+const prettier = require("prettier");
+const code = "(add 1 2)";
+prettier.format(code, {
+  parser: "lisp",
+  plugins: ["."]
+});
+```
+
+This will resolve a plugin relative to the current working direcrory.

index.js

@@ -1,15 +1,16 @@
 "use strict";
 
-const comments = require("./src/comments");
+const comments = require("./src/main/comments");
 const version = require("./package.json").version;
-const printAstToDoc = require("./src/printer").printAstToDoc;
-const util = require("./src/util");
-const printDocToString = require("./src/doc-printer").printDocToString;
-const normalizeOptions = require("./src/options").normalize;
-const parser = require("./src/parser");
-const printDocToDebug = require("./src/doc-debug").printDocToDebug;
-const config = require("./src/resolve-config");
-const getSupportInfo = require("./src/support").getSupportInfo;
+const printAstToDoc = require("./src/main/ast-to-doc");
+const util = require("./src/common/util");
+const doc = require("./src/doc");
+const printDocToString = doc.printer.printDocToString;
+const printDocToDebug = doc.debug.printDocToDebug;
+const normalizeOptions = require("./src/common/options").normalize;
+const parser = require("./src/main/parser");
+const config = require("./src/config/resolve-config");
+const getSupportInfo = require("./src/common/support").getSupportInfo;
 const docblock = require("jest-docblock");
 
 function guessLineEnding(text) {
@@ -385,6 +386,8 @@ module.exports = {
     }
   },
 
+  doc,
+
   resolveConfig: config.resolveConfig,
   clearConfigCache: config.clearCache,
 

package.json

@@ -47,6 +47,7 @@
     "postcss-values-parser": "1.3.1",
     "remark-frontmatter": "1.1.0",
     "remark-parse": "4.0.0",
+    "resolve": "1.5.0",
     "semver": "5.4.1",
     "string-width": "2.1.1",
     "typescript": "2.6.2",
@@ -70,8 +71,8 @@
     "prettier": "1.9.2",
     "prettylint": "1.0.0",
     "rimraf": "2.6.2",
-    "rollup": "0.41.6",
-    "rollup-plugin-commonjs": "7.0.2",
+    "rollup": "0.47.6",
+    "rollup-plugin-commonjs": "8.2.6",
     "rollup-plugin-json": "2.1.1",
     "rollup-plugin-node-builtins": "2.0.0",
     "rollup-plugin-node-globals": "1.1.0",

scripts/build/build-docs.js

@@ -4,19 +4,12 @@
 
 const path = require("path");
 const shell = require("shelljs");
+const parsers = require("./parsers");
 
 const rootDir = path.join(__dirname, "..", "..");
 const docs = path.join(rootDir, "website/static/lib");
-const parsers = [
-  "babylon",
-  "flow",
-  "typescript",
-  "graphql",
-  "postcss",
-  "parse5",
-  "markdown",
-  "vue"
-];
+
+const stripLanguageDirectory = parserPath => parserPath.replace(/.*\//, "");
 
 function pipe(string) {
   return new shell.ShellString(string);
@@ -25,6 +18,8 @@ function pipe(string) {
 const isPullRequest = process.env.PULL_REQUEST === "true";
 const prettierPath = isPullRequest ? "dist" : "node_modules/prettier/";
 
+const parserPaths = parsers.map(stripLanguageDirectory);
+
 // --- Build prettier for PR ---
 
 if (isPullRequest) {
@@ -44,21 +39,19 @@ shell.exec(
   `node_modules/babel-cli/bin/babel.js ${docs}/index.js --out-file ${docs}/index.js --presets=es2015`
 );
 
-shell.echo("Bundling docs babylon...");
 shell.exec(
   `rollup -c scripts/build/rollup.docs.config.js --environment filepath:parser-babylon.js -i ${prettierPath}/parser-babylon.js`
 );
 shell.exec(
   `node_modules/babel-cli/bin/babel.js ${docs}/parser-babylon.js --out-file ${docs}/parser-babylon.js --presets=es2015`
 );
 
-for (const parser of parsers) {
-  if (parser === "babylon") {
+for (const parser of parserPaths) {
+  if (parser.endsWith("babylon")) {
     continue;
   }
-  shell.echo(`Bundling docs ${parser}...`);
   shell.exec(
-    `rollup -c scripts/build/rollup.docs.config.js --environment filepath:parser-${parser}.js -i ${prettierPath}/parser-${parser}.js`
+    `rollup -c scripts/build/rollup.docs.config.js --environment filepath:${parser}.js -i ${prettierPath}/${parser}.js`
   );
 }
 

scripts/build/build.js

@@ -5,19 +5,10 @@
 const path = require("path");
 const pkg = require("../../package.json");
 const formatMarkdown = require("../../website/static/markdown");
+const parsers = require("./parsers");
 const shell = require("shelljs");
 
 const rootDir = path.join(__dirname, "..", "..");
-const parsers = [
-  "babylon",
-  "flow",
-  "typescript",
-  "graphql",
-  "postcss",
-  "parse5",
-  "markdown",
-  "vue"
-];
 
 process.env.PATH += path.delimiter + path.join(rootDir, "node_modules", ".bin");
 
@@ -32,30 +23,26 @@ shell.rm("-Rf", "dist/");
 
 // --- Lib ---
 
-shell.echo("Bundling lib index...");
 shell.exec("rollup -c scripts/build/rollup.index.config.js");
 
-shell.echo("Bundling lib bin...");
 shell.exec("rollup -c scripts/build/rollup.bin.config.js");
 shell.chmod("+x", "./dist/bin/prettier.js");
 
-shell.echo("Bundling lib third-party...");
 shell.exec("rollup -c scripts/build/rollup.third-party.config.js");
 
 for (const parser of parsers) {
-  if (parser === "postcss") {
+  if (parser.endsWith("postcss")) {
     continue;
   }
-  shell.echo(`Bundling lib ${parser}...`);
   shell.exec(
     `rollup -c scripts/build/rollup.parser.config.js --environment parser:${parser}`
   );
 }
 
-shell.echo("Bundling lib postcss...");
+shell.echo("\nsrc/language-css/parser-postcss.js → dist/parser-postcss.js");
 // PostCSS has dependency cycles and won't work correctly with rollup :(
 shell.exec(
-  "webpack --hide-modules src/parser-postcss.js dist/parser-postcss.js"
+  "webpack --hide-modules src/language-css/parser-postcss.js dist/parser-postcss.js"
 );
 // Prepend module.exports =
 const content = shell.cat("dist/parser-postcss.js").stdout;

scripts/build/parsers.js

@@ -0,0 +1,12 @@
+"use strict";
+
+module.exports = [
+  "language-js/parser-babylon",
+  "language-js/parser-flow",
+  "language-js/parser-typescript",
+  "language-graphql/parser-graphql",
+  "language-css/parser-postcss",
+  "language-html/parser-parse5",
+  "language-markdown/parser-markdown",
+  "language-vue/parser-vue"
+];

scripts/build/rollup.bin.config.js

@@ -24,9 +24,9 @@ export default Object.assign(baseConfig, {
     "assert",
     "util",
     "events",
-    path.resolve("src/third-party.js")
+    path.resolve("src/common/third-party.js")
   ],
   paths: {
-    [path.resolve("src/third-party.js")]: "../third-party"
+    [path.resolve("src/common/third-party.js")]: "../third-party"
   }
 });

scripts/build/rollup.index.config.js

@@ -8,7 +8,7 @@ import * as path from "path";
 const external = ["assert"];
 
 if (process.env.BUILD_TARGET !== "website") {
-  external.push(path.resolve("src/third-party.js"));
+  external.push(path.resolve("src/common/third-party.js"));
 }
 
 export default Object.assign(baseConfig, {
@@ -25,6 +25,6 @@ export default Object.assign(baseConfig, {
   ],
   external,
   paths: {
-    [path.resolve("src/third-party.js")]: "./third-party"
+    [path.resolve("src/common/third-party.js")]: "./third-party"
   }
 });

scripts/build/rollup.parser.config.js

@@ -4,15 +4,16 @@ import commonjs from "rollup-plugin-commonjs";
 import json from "rollup-plugin-json";
 import replace from "rollup-plugin-replace";
 import uglify from "uglify-es";
+import path from "path";
 
 const parser = process.env.parser;
 
 export default Object.assign(baseConfig, {
-  entry: "src/parser-" + parser + ".js",
-  dest: "dist/parser-" + parser + ".js",
+  entry: "src/" + parser + ".js",
+  dest: "dist/" + path.basename(parser) + ".js",
   format: "cjs",
   plugins: [
-    parser === "typescript"
+    parser.endsWith("typescript")
       ? replace({
           "exports.Syntax =": "1,",
           include: "node_modules/typescript-eslint-parser/parser.js"
@@ -21,7 +22,7 @@ export default Object.assign(baseConfig, {
     // In flow-parser 0.59.0 there's a dynamic require: `require(s8)` which not
     // supported by rollup-plugin-commonjs, so we have to replace the variable
     // by its value before bundling.
-    parser === "flow"
+    parser.endsWith("flow")
       ? replace({
           "require(s8)": 'require("fs")',
           include: "node_modules/flow-parser/flow_parser.js"
@@ -50,5 +51,5 @@ export default Object.assign(baseConfig, {
     "os",
     "crypto"
   ],
-  useStrict: parser !== "flow"
+  useStrict: !parser.endsWith("flow")
 });