Skip to content

Commit

Permalink
chore: rewrite scripts/ with javascript
Browse files Browse the repository at this point in the history
Goals of this rewrite:

- Better portability of scripts to anywhere we can run `node` and no
  reliance on `make` or other tools
- More code inside `scripts/` and less inside of CI workflows to allow
  for easier local testing
- Reuse the same scripts for similar workflows (eg `publish` and
  `smoke-publish`)

Future goals:

- Allow testing of `scripts`

Some highlights of the new scripts:

`scripts/create-node-pr.js`

- This now operates on cloned copy of the base repo that is pushed to
  our fork, so we no longer need to manually sync our fork.
- Uses the published registry tarball for as much of the PR as possible,
  falling back to including local files from source. This will enable
  for easier migration to eventually only using the tarball contents.

`scripts/publish.js`

- Replaces `make publish`
- Will publish all workspaces that need to be published, before
  publishing the CLI.
- Makes running tests an optional part of the script, since CI now runs
  the tests.
- Can optionally only pack the tarball, for use in CI to test installing
  the tarball.

`scripts/util.js`

- Shared utilities for spawning commands including helpers for `npm` and
  `gh`.
- Common interface for running, parsing args, logging, and debugging
  of scripts.
  • Loading branch information
lukekarrys authored and wraithgar committed Nov 1, 2022
1 parent 0c5834e commit 9709ae6
Show file tree
Hide file tree
Showing 46 changed files with 668 additions and 436 deletions.
6 changes: 0 additions & 6 deletions .eslintrc.local.json
Original file line number Diff line number Diff line change
@@ -1,16 +1,10 @@
{
"ignorePatterns": [
"docs/",
"workspaces/*"
],
"rules": {
"no-shadow": "off",
"no-console": "error"
},
"overrides": [{
"files": [
"scripts/**",
"bin/**",
"test/**"
],
"rules": {
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/ci-libnpmaccess.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,6 @@ jobs:
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Link
if: matrix
run: node . link -f --ignore-scripts
- name: Rebuild cmark-gfm
run: node . rebuild cmark-gfm
- name: Add Problem Matcher
run: echo "::add-matcher::.github/matchers/tap.json"
- name: Test
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/ci-libnpmdiff.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,6 @@ jobs:
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Link
if: matrix
run: node . link -f --ignore-scripts
- name: Rebuild cmark-gfm
run: node . rebuild cmark-gfm
- name: Add Problem Matcher
run: echo "::add-matcher::.github/matchers/tap.json"
- name: Test
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/ci-libnpmexec.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,6 @@ jobs:
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Link
if: matrix
run: node . link -f --ignore-scripts
- name: Rebuild cmark-gfm
run: node . rebuild cmark-gfm
- name: Add Problem Matcher
run: echo "::add-matcher::.github/matchers/tap.json"
- name: Test
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/ci-libnpmfund.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,6 @@ jobs:
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Link
if: matrix
run: node . link -f --ignore-scripts
- name: Rebuild cmark-gfm
run: node . rebuild cmark-gfm
- name: Add Problem Matcher
run: echo "::add-matcher::.github/matchers/tap.json"
- name: Test
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/ci-libnpmhook.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,6 @@ jobs:
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Link
if: matrix
run: node . link -f --ignore-scripts
- name: Rebuild cmark-gfm
run: node . rebuild cmark-gfm
- name: Add Problem Matcher
run: echo "::add-matcher::.github/matchers/tap.json"
- name: Test
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/ci-libnpmorg.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,6 @@ jobs:
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Link
if: matrix
run: node . link -f --ignore-scripts
- name: Rebuild cmark-gfm
run: node . rebuild cmark-gfm
- name: Add Problem Matcher
run: echo "::add-matcher::.github/matchers/tap.json"
- name: Test
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/ci-libnpmpack.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,6 @@ jobs:
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Link
if: matrix
run: node . link -f --ignore-scripts
- name: Rebuild cmark-gfm
run: node . rebuild cmark-gfm
- name: Add Problem Matcher
run: echo "::add-matcher::.github/matchers/tap.json"
- name: Test
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/ci-libnpmpublish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,6 @@ jobs:
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Link
if: matrix
run: node . link -f --ignore-scripts
- name: Rebuild cmark-gfm
run: node . rebuild cmark-gfm
- name: Add Problem Matcher
run: echo "::add-matcher::.github/matchers/tap.json"
- name: Test
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/ci-libnpmsearch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,6 @@ jobs:
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Link
if: matrix
run: node . link -f --ignore-scripts
- name: Rebuild cmark-gfm
run: node . rebuild cmark-gfm
- name: Add Problem Matcher
run: echo "::add-matcher::.github/matchers/tap.json"
- name: Test
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/ci-libnpmteam.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,6 @@ jobs:
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Link
if: matrix
run: node . link -f --ignore-scripts
- name: Rebuild cmark-gfm
run: node . rebuild cmark-gfm
- name: Add Problem Matcher
run: echo "::add-matcher::.github/matchers/tap.json"
- name: Test
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/ci-libnpmversion.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,6 @@ jobs:
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Link
if: matrix
run: node . link -f --ignore-scripts
- name: Rebuild cmark-gfm
run: node . rebuild cmark-gfm
- name: Add Problem Matcher
run: echo "::add-matcher::.github/matchers/tap.json"
- name: Test
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/ci-npmcli-arborist.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,6 @@ jobs:
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Link
if: matrix
run: node . link -f --ignore-scripts
- name: Rebuild cmark-gfm
run: node . rebuild cmark-gfm
- name: Add Problem Matcher
run: echo "::add-matcher::.github/matchers/tap.json"
- name: Test
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/ci-npmcli-docs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -111,11 +111,6 @@ jobs:
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Link
if: matrix
run: node . link -f --ignore-scripts
- name: Rebuild cmark-gfm
run: node . rebuild cmark-gfm
- name: Add Problem Matcher
run: echo "::add-matcher::.github/matchers/tap.json"
- name: Test
Expand Down
21 changes: 5 additions & 16 deletions .github/workflows/ci-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -177,11 +177,6 @@ jobs:
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Link
if: matrix
run: node . link -f --ignore-scripts
- name: Rebuild cmark-gfm
run: node . rebuild cmark-gfm
- name: Add Problem Matcher
run: echo "::add-matcher::.github/matchers/tap.json"
- name: Test
Expand Down Expand Up @@ -284,18 +279,12 @@ jobs:
run: |
NPM_VERSION="$(node . --version)-$GITHUB_SHA.0"
node . version $NPM_VERSION --ignore-scripts
node . run resetdeps
git clean -fd
node . ls --omit=dev >/dev/null
node . prune --omit=dev --no-save --no-audit --no-fund
node scripts/git-dirty.js
node . pack --pack-destination=$RUNNER_TEMP
node . install -g $RUNNER_TEMP/npm-$NPM_VERSION.tgz
node scripts/publish.js --pack-destination=$RUNNER_TEMP
node . install --global $RUNNER_TEMP/npm-$NPM_VERSION.tgz
node . install -w smoke-tests --ignore-scripts --no-audit --no-fund
rm -rf {lib,bin,index.js}
# this one should be npm since we explicitly installed our packed
# tarball globally and the next test will make sure our the new
# globally installed version contains the git sha
node scripts/remove-files.js
# call installed npm instead of local source since we are testing
# the packed tarball that we just installed globally
SMOKE_PUBLISH_NPM=1 npm test -w smoke-tests --ignore-scripts
- name: Conclude Check
uses: LouisBrunner/[email protected]
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/ci-smoke-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -118,11 +118,6 @@ jobs:
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Link
if: matrix
run: node . link -f --ignore-scripts
- name: Rebuild cmark-gfm
run: node . rebuild cmark-gfm
- name: Add Problem Matcher
run: echo "::add-matcher::.github/matchers/tap.json"
- name: Test
Expand Down
5 changes: 0 additions & 5 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -122,11 +122,6 @@ jobs:
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Link
if: matrix
run: node . link -f --ignore-scripts
- name: Rebuild cmark-gfm
run: node . rebuild cmark-gfm
- name: Add Problem Matcher
run: echo "::add-matcher::.github/matchers/tap.json"
- name: Test
Expand Down
96 changes: 0 additions & 96 deletions .github/workflows/create-cli-deps-pr.yml

This file was deleted.

49 changes: 49 additions & 0 deletions .github/workflows/create-node-pr.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
# This file is automatically added by @npmcli/template-oss. Do not edit.

name: "Create Node PR"

on:
workflow_dispatch:
inputs:
spec:
description: "The npm spec to create the PR from"
required: true
default: 'latest'
dryRun:
description: "Setting this to anything will run all the steps except opening the PR"

jobs:
create-pull-request:
name: Create Node PR
if: github.repository_owner == 'npm'
runs-on: ubuntu-latest
defaults:
run:
shell: bash
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Setup Git User
run: |
git config --global user.email "[email protected]"
git config --global user.name "npm CLI robot"
- name: Setup Node
uses: actions/setup-node@v3
with:
node-version: 18.x
cache: npm
- name: Reset Deps
run: node . run resetdeps
- name: Checkout Node
uses: actions/checkout@v3
with:
token: ${{ secrets.NODE_PULL_REQUEST_TOKEN }}
repository: nodejs/node
fetch-depth: 0
path: node
- name: Create Node Pull Request
env:
GITHUB_TOKEN: ${{ secrets.NODE_PULL_REQUEST_TOKEN }}
run: |
DRY_RUN=$([ -z "${{ inputs.dryRun }}" ] && echo "" || echo "--dry-run")
node scripts/create-node-pr.js "${{ inputs.spec }}" "$DRY_RUN"
3 changes: 0 additions & 3 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@
!/AUTHORS
!/bin/
!/CHANGELOG*
!/changelogs/
!/CODE_OF_CONDUCT.md
!/configure
!/CONTRIBUTING.md
Expand All @@ -27,8 +26,6 @@
!/index.js
!/lib/
!/LICENSE*
!/make.bat
!/Makefile
!/map.js
!/node_modules/
/node_modules/.bin/
Expand Down
Loading

0 comments on commit 9709ae6

Please sign in to comment.