Add 'npm exec', port npx to use it directly

This removes libnpx, and adds a new command, 'npm exec', which
implements the functionality.  As of this change going live, we are
dropping support for the standalone top-level package 'npx'.

Not all of the functionality of the old version of npx is maintained.

The shell fallback functionality is dropped.  It's insecure, and not
something we want to support or encourage.  If anyone wants it, they
can hack up their .bashrc file themselves.

--no-install is not supported.  If the package is not found locally, it
is installed in a predictable location in the cache, rather than
failing.  This is something we might want to review, as automatically
installing in the case of misspellings may be a security footgun.

--ignore-existing is dropped.  Existing packages are always given
priority.

--quiet or -q can be accomplished by using the --silent npm option, so
it's also dropped.

--npm option is dropped.  npx will always use the npm that it ships
with.

--node-arg is dropped.  There are other ways to set node options via
environment variables in the Node.js versions we support.

--always-spawn is dropped.  npx will always spawn a child process to
execute commands.

The --shell option can be accomplished by using the --script-shell npm
option.

--version and --help are just passed through to npm.

As an added bonus, I noticed that the files in `bin/` were not getting
run.  So now we have full coverage for npm-cli.js and npx-cli.js.

PR-URL: npm#1588
Credit: @isaacs
Close: npm#1588
Reviewed-by: @ruyadorno

.gitignore

@@ -1,7 +1,6 @@
 *.swp
 npm-debug.log
 .nyc_output/
-/test/bin
 /test/output.log
 /test/*/*/node_modules
 /test/packages/npm-test-depends-on-spark/which-spark.log

Makefile

@@ -9,8 +9,7 @@ markdowns = $(shell find docs -name '*.md' | grep -v 'index') README.md
 cli_mandocs = $(shell find docs/content/cli-commands -name '*.md' \
                |sed 's|.md|.1|g' \
                |sed 's|docs/content/cli-commands/|man/man1/|g' ) \
-               man/man1/npm-README.1 \
-               man/man1/npx.1
+               man/man1/npm-README.1
 
 files_mandocs = $(shell find docs/content/configuring-npm -name '*.md' \
                |sed 's|.md|.5|g' \
@@ -84,9 +83,6 @@ man/man1/%.1: docs/content/cli-commands/%.md scripts/docs-build.js package.json
 	@[ -d man/man1 ] || mkdir -p man/man1
 	node scripts/docs-build.js $< $@
 
-man/man1/npx.1: node_modules/libnpx/libnpx.1
-	cat $< | sed s/libnpx/npx/ > $@
-
 man/man5/npm-json.5: man/man5/package.json.5
 	cp $< $@
 

bin/npx-cli.js

@@ -1,8 +1,30 @@
 #!/usr/bin/env node
 
-const npx = require('libnpx')
-const path = require('path')
+const cli = require('../lib/cli.js')
 
-const NPM_PATH = path.join(__dirname, 'npm-cli.js')
+// run the resulting command as `npm exec ...args`
+process.argv[1] = require.resolve('./npm-cli.js')
+process.argv.splice(2, 0, 'exec')
 
-npx(npx.parseArgs(process.argv, NPM_PATH))
+// break out of loop when we find a positional argument or --
+// If we find a positional arg, we shove -- in front of it, and
+// let the normal npm cli handle the rest.
+let i
+for (i = 3; i < process.argv.length; i++) {
+  const arg = process.argv[i]
+  if (arg === '--') {
+    break
+  } else if (/^-/.test(arg)) {
+    // `--package foo` treated the same as `--package=foo`
+    if (!arg.includes('=')) {
+      i++
+    }
+    continue
+  } else {
+    // found a positional arg, put -- in front of it, and we're done
+    process.argv.splice(i, 0, '--')
+    break
+  }
+}
+
+cli(process)

docs/content/cli-commands/npm-exec.md

@@ -0,0 +1,148 @@
+---
+section: cli-commands
+title: npm-exec
+description: Run a command from a local or remote npm package
+---
+
+# npm-exec(1)
+
+## Run a command from a local or remote npm package
+
+### Synopsis
+
+```bash
+npm exec -- <pkg>[@<version>] [args...]
+npm exec -p <pkg>[@<version>] -- <cmd> [args...]
+npm exec -c '<cmd> [args...]'
+npm exec -p foo -c '<cmd> [args...]'
+
+npx <pkg>[@<specifier>] [args...]
+npx -p <pkg>[@<specifier>] <cmd> [args...]
+npx -c '<cmd> [args...]'
+npx -p <pkg>[@<specifier>] -c '<cmd> [args...]'
+
+alias: npm x, npx
+
+-p <pkg> --package=<pkg> (may be specified multiple times)
+-c <cmd> --call=<cmd> (may not be mixed with positional arguments)
+```
+
+### Description
+
+This command allows you to run an arbitrary command from an npm package
+(either one installed locally, or fetched remotely), in a similar context
+as running it via `npm run`.
+
+Whatever packages are specified by the `--package` or `-p` option will be
+provided in the `PATH` of the executed command, along with any locally
+installed package executables.  The `--package` or `-p` option may be
+specified multiple times, to execute the supplied command in an environment
+where all specified packages are available.
+
+If any requested packages are not present in the local project
+dependencies, then they are installed to a folder in the npm cache, which
+is added to the `PATH` environment variable in the executed process.
+Package names provided without a specifier will be matched with whatever
+version exists in the local project.  Package names with a specifier will
+only be considered a match if they have the exact same name and version as
+the local dependency.
+
+If no `-c` or `--call` option is provided, then the positional arguments
+are used to generate the command string.  If no `-p` or `--package` options
+are provided, then npm will attempt to determine the executable name from
+the package specifier provided as the first positional argument according
+to the following heuristic:
+
+- If the package has a single entry in its `bin` field in `package.json`,
+  then that command will be used.
+- If the package has multiple `bin` entries, and one of them matches the
+  unscoped portion of the `name` field, then that command will be used.
+- If this does not result in exactly one option (either because there are
+  no bin entries, or none of them match the `name` of the package), then
+  `npm exec` exits with an error.
+
+To run a binary _other than_ the named binary, specify one or more
+`--package` options, which will prevent npm from inferring the package from
+the first command argument.
+
+### `npx` vs `npm exec`
+
+When run via the `npx` binary, all flags and options *must* be set prior to
+any positional arguments.  When run via `npm exec`, a double-hyphen `--`
+flag can be used to suppress npm's parsing of switches and options that
+should be sent to the executed command.
+
+For example:
+
+```
+$ npx foo@latest bar --package=@npmcli/foo
+```
+
+In this case, npm will resolve the `foo` package name, and run the
+following command:
+
+```
+$ foo bar --package=@npmcli/foo
+```
+
+Since the `--package` option comes _after_ the positional arguments, it is
+treated as an argument to the executed command.
+
+In contrast, due to npm's argument parsing logic, running this command is
+different:
+
+```
+$ npm exec foo@latest bar --package=@npmcli/foo
+```
+
+In this case, npm will parse the `--package` option first, resolving the
+`@npmcli/foo` package.  Then, it will execute the following command in that
+context:
+
+```
+$ foo@latest bar
+```
+
+The double-hyphen character is recommended to explicitly tell npm to stop
+parsing command line options and switches.  The following command would
+thus be equivalent to the `npx` command above:
+
+```
+$ npm exec -- foo@latest bar --package=@npmcli/foo
+```
+
+### Examples
+
+Run the version of `tap` in the local dependencies, with the provided
+arguments:
+
+```
+$ npm exec -- tap --bail test/foo.js
+$ npx tap --bail test/foo.js
+```
+
+Run a command _other than_ the command whose name matches the package name
+by specifying a `--package` option:
+
+```
+$ npm exec --package=foo -- bar --bar-argument
+# ~ or ~
+$ npx --package=foo bar --bar-argument
+```
+
+Run an arbitrary shell script, in the context of the current project:
+
+```
+$ npm x -c 'eslint && say "hooray, lint passed"'
+$ npx -c 'eslint && say "hooray, lint passed"'
+```
+
+### See Also
+
+* [npm run-script](/cli-commands/run-script)
+* [npm scripts](/using-npm/scripts)
+* [npm test](/cli-commands/test)
+* [npm start](/cli-commands/start)
+* [npm restart](/cli-commands/restart)
+* [npm stop](/cli-commands/stop)
+* [npm config](/cli-commands/config)

docs/content/cli-commands/npx.md

@@ -0,0 +1,148 @@
+---
+section: cli-commands
+title: npx
+description: Run a command from a local or remote npm package
+---
+
+# npx(1)
+
+## Run a command from a local or remote npm package
+
+### Synopsis
+
+```bash
+npm exec -- <pkg>[@<version>] [args...]
+npm exec -p <pkg>[@<version>] -- <cmd> [args...]
+npm exec -c '<cmd> [args...]'
+npm exec -p foo -c '<cmd> [args...]'
+
+npx <pkg>[@<specifier>] [args...]
+npx -p <pkg>[@<specifier>] <cmd> [args...]
+npx -c '<cmd> [args...]'
+npx -p <pkg>[@<specifier>] -c '<cmd> [args...]'
+
+alias: npm x, npx
+
+-p <pkg> --package=<pkg> (may be specified multiple times)
+-c <cmd> --call=<cmd> (may not be mixed with positional arguments)
+```
+
+### Description
+
+This command allows you to run an arbitrary command from an npm package
+(either one installed locally, or fetched remotely), in a similar context
+as running it via `npm run`.
+
+Whatever packages are specified by the `--package` or `-p` option will be
+provided in the `PATH` of the executed command, along with any locally
+installed package executables.  The `--package` or `-p` option may be
+specified multiple times, to execute the supplied command in an environment
+where all specified packages are available.
+
+If any requested packages are not present in the local project
+dependencies, then they are installed to a folder in the npm cache, which
+is added to the `PATH` environment variable in the executed process.
+Package names provided without a specifier will be matched with whatever
+version exists in the local project.  Package names with a specifier will
+only be considered a match if they have the exact same name and version as
+the local dependency.
+
+If no `-c` or `--call` option is provided, then the positional arguments
+are used to generate the command string.  If no `-p` or `--package` options
+are provided, then npm will attempt to determine the executable name from
+the package specifier provided as the first positional argument according
+to the following heuristic:
+
+- If the package has a single entry in its `bin` field in `package.json`,
+  then that command will be used.
+- If the package has multiple `bin` entries, and one of them matches the
+  unscoped portion of the `name` field, then that command will be used.
+- If this does not result in exactly one option (either because there are
+  no bin entries, or none of them match the `name` of the package), then
+  `npm exec` exits with an error.
+
+To run a binary _other than_ the named binary, specify one or more
+`--package` options, which will prevent npm from inferring the package from
+the first command argument.
+
+### `npx` vs `npm exec`
+
+When run via the `npx` binary, all flags and options *must* be set prior to
+any positional arguments.  When run via `npm exec`, a double-hyphen `--`
+flag can be used to suppress npm's parsing of switches and options that
+should be sent to the executed command.
+
+For example:
+
+```
+$ npx foo@latest bar --package=@npmcli/foo
+```
+
+In this case, npm will resolve the `foo` package name, and run the
+following command:
+
+```
+$ foo bar --package=@npmcli/foo
+```
+
+Since the `--package` option comes _after_ the positional arguments, it is
+treated as an argument to the executed command.
+
+In contrast, due to npm's argument parsing logic, running this command is
+different:
+
+```
+$ npm exec foo@latest bar --package=@npmcli/foo
+```
+
+In this case, npm will parse the `--package` option first, resolving the
+`@npmcli/foo` package.  Then, it will execute the following command in that
+context:
+
+```
+$ foo@latest bar
+```
+
+The double-hyphen character is recommended to explicitly tell npm to stop
+parsing command line options and switches.  The following command would
+thus be equivalent to the `npx` command above:
+
+```
+$ npm exec -- foo@latest bar --package=@npmcli/foo
+```
+
+### Examples
+
+Run the version of `tap` in the local dependencies, with the provided
+arguments:
+
+```
+$ npm exec -- tap --bail test/foo.js
+$ npx tap --bail test/foo.js
+```
+
+Run a command _other than_ the command whose name matches the package name
+by specifying a `--package` option:
+
+```
+$ npm exec --package=foo -- bar --bar-argument
+# ~ or ~
+$ npx --package=foo bar --bar-argument
+```
+
+Run an arbitrary shell script, in the context of the current project:
+
+```
+$ npm x -c 'eslint && say "hooray, lint passed"'
+$ npx -c 'eslint && say "hooray, lint passed"'
+```
+
+### See Also
+
+* [npm run-script](/cli-commands/run-script)
+* [npm scripts](/using-npm/scripts)
+* [npm test](/cli-commands/test)
+* [npm start](/cli-commands/start)
+* [npm restart](/cli-commands/restart)
+* [npm stop](/cli-commands/stop)
+* [npm config](/cli-commands/config)

lib/config/cmd-list.js

@@ -19,7 +19,8 @@ const shorthands = {
   v: 'view',
   run: 'run-script',
   'clean-install': 'ci',
-  'clean-install-test': 'cit'
+  'clean-install-test': 'cit',
+  x: 'exec'
 }
 
 const affordances = {
@@ -119,7 +120,8 @@ const cmdList = [
   'restart',
   'run-script',
   'completion',
-  'doctor'
+  'doctor',
+  'exec'
 ]
 
 const plumbing = ['birthday']