Skip to content

Commit

Permalink
repo: migrate away from pnpm (ethereum-optimism#11395)
Browse files Browse the repository at this point in the history
Migrate to just from pnpm. This is continued tech debt
cleanup away from `pnpm` and typescript that is no longer
used.
  • Loading branch information
tynes authored Aug 9, 2024
1 parent bf4693e commit 770cbc6
Show file tree
Hide file tree
Showing 17 changed files with 69 additions and 3,692 deletions.
68 changes: 7 additions & 61 deletions .circleci/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -188,38 +188,18 @@ jobs:
name: "Check L1 geth version"
command: ./ops/scripts/geth-version-checker.sh || (echo "geth version is wrong, update ci-builder"; false)
- install-contracts-dependencies
- restore_cache:
name: Restore PNPM Package Cache
keys:
- pnpm-packages-v2-{{ checksum "pnpm-lock.yaml" }}
- restore_cache:
name: Restore Go modules cache
key: gomod-{{ checksum "go.sum" }}
# Fetch node_modules into the pnpm store
# This will cache node_modules based on pnpm-lock so other steps can instantly install them with `pnpm install --prefer-offline`
# --prefer-offline installs node_modules instantly by just reading from cache if it exists rather than fetching from network
# when installing node_modules pnpm simply adds symlinks instead of copying the files which is why it is pretty much instant to run --prefer-offline
# this allows a caching strategy of only checking pnpm-lockfile so we don't have to keep it in sync with our packages
# For more information see https://pnpm.io/cli/fetch
- run:
name: Fetch dependencies
command: pnpm fetch --frozen-lockfile --prefer-offline
- save_cache:
name: Save PNPM Package Cache
key: pnpm-packages-v2-{{ checksum "pnpm-lock.yaml" }}
paths:
- "node_modules"
- run:
name: Install dependencies
command: pnpm install
- run:
name: print forge version
command: forge --version
- run:
name: Build monorepo
name: Build contracts
environment:
FOUNDRY_PROFILE: ci
command: pnpm build
command: just build
working_directory: packages/contracts-bedrock
- run:
name: Generate L2OO allocs
command: DEVNET_L2OO="true" make devnet-allocs
Expand All @@ -244,12 +224,9 @@ jobs:
- persist_to_workspace:
root: "."
paths:
- "packages/**/dist"
- "packages/contracts-bedrock/cache"
- "packages/contracts-bedrock/artifacts"
- "packages/contracts-bedrock/forge-artifacts"
- "packages/contracts-bedrock/tsconfig.tsbuildinfo"
- "packages/contracts-bedrock/tsconfig.build.tsbuildinfo"
- ".devnet/allocs-l1.json"
- ".devnet/allocs-l2-delta.json"
- ".devnet/allocs-l2-ecotone.json"
Expand Down Expand Up @@ -580,21 +557,12 @@ jobs:
resource_class: xlarge
steps:
- checkout
- install-contracts-dependencies
- restore_cache:
name: Restore PNPM Package Cache
keys:
- pnpm-packages-v2-{{ checksum "pnpm-lock.yaml" }}
- attach_workspace: { at: "." }
- install-contracts-dependencies
- check-changed:
patterns: contracts-bedrock,op-node
- setup_remote_docker:
docker_layer_caching: true
# populate node modules from the cache
- run:
name: Install dependencies
command: pnpm install
working_directory: packages/contracts-bedrock
- run:
name: forge version
command: forge --version
Expand Down Expand Up @@ -692,14 +660,7 @@ jobs:
resource_class: medium
steps:
- checkout
- restore_cache:
name: Restore PNPM Package Cache
keys:
- pnpm-packages-v2-{{ checksum "pnpm-lock.yaml" }}
- attach_workspace: { at: "." }
- run:
name: Install dependencies
command: pnpm install
- check-changed:
patterns: contracts-bedrock
- run:
Expand Down Expand Up @@ -1156,26 +1117,11 @@ jobs:
echo 'export PATH=$HOME/.foundry/bin:$PATH' >> $BASH_ENV
source $HOME/.bashrc
forge --version
- run:
name: Install NVM
command: |
curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.7/install.sh | bash
source ~/.bashrc
nvm --version
- run:
name: Install Node
command: |
nvm install
nvm use && node --version && npm --version
- run:
name: Install Just
command: |
curl --proto '=https' --tlsv1.2 -sSf https://just.systems/install.sh | bash -s -- --to $HOME/bin
echo 'export PATH="${PATH}:$HOME/bin"' >> $BASH_ENV
- run:
name: Install pnpm
command: |
npm i pnpm --global
- install-contracts-dependencies
- attach_workspace:
at: "."
Expand Down Expand Up @@ -1204,9 +1150,9 @@ jobs:
docker tag "$IMAGE_BASE_PREFIX/op-challenger:<<pipeline.git.revision>>" "$IMAGE_BASE_PREFIX/op-challenger:devnet"
docker tag "$IMAGE_BASE_PREFIX/da-server:<<pipeline.git.revision>>" "$IMAGE_BASE_PREFIX/da-server:devnet"
- run:
name: pnpm install and build
command: |
pnpm install && pnpm build
name: Build contracts
working_directory: packages/contracts-bedrock
command: just build
- run:
name: Bring up the stack
command: |
Expand Down
24 changes: 0 additions & 24 deletions .github/actions/setup/action.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,29 +3,5 @@ description: Common setup steps used by our workflows
runs:
using: composite
steps:
- name: Setup pnpm
uses: pnpm/action-setup@v3
with:
version: 9
- name: Setup node
uses: actions/setup-node@v3
with:
node-version-file: .nvmrc
registry-url: https://registry.npmjs.org
cache: pnpm

- name: Setup foundry
uses: foundry-rs/foundry-toolchain@v1

- name: Install node dependencies
shell: bash
run: pnpm install --frozen-lockfile
- name: Derive appropriate SHAs for base and head for `nx affected` commands
uses: nrwl/nx-set-shas@v3
with:
main-branch-name: "develop"
- run: |
echo "nx using following shas:"
echo "BASE: ${{ env.NX_BASE }}"
echo "HEAD: ${{ env.NX_HEAD }}"
shell: bash
1 change: 0 additions & 1 deletion .nvmrc

This file was deleted.

20 changes: 6 additions & 14 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Interactions within this repository are subject to a [Code of Conduct](https://g
| [go](https://go.dev/) | `^1.21` | `go version` |
| [node](https://nodejs.org/en/) | `^20` | `node --version` |
| [nvm](https://github.com/nvm-sh/nvm) | `^0.39` | `nvm --version` |
| [pnpm](https://pnpm.io/installation) | `^8` | `pnpm --version` |
| [just](https://github.com/casey/just) | `^1.34.0`| `just --version` |
| [foundry](https://github.com/foundry-rs/foundry#installation) | `^0.2.0` | `forge --version` |
| [make](https://linux.die.net/man/1/make) | `^3` | `make --version` |
| [jq](https://github.com/jqlang/jq) | `^1.6` | `jq --version` |
Expand All @@ -46,7 +46,7 @@ You can use [`nvm`](https://github.com/nvm-sh/nvm) to manage multiple versions o

`foundry` is updated frequently and occasionally contains breaking changes.
This repository pins a specific version of `foundry` inside of [`versions.json`](./versions.json).
Use the command `pnpm update:foundry` at the root of the monorepo to make sure that your version of `foundry` is the same as the one currently being used in CI.
Use the command `just update-foundry` at the root of the monorepo to make sure that your version of `foundry` is the same as the one currently being used in CI.

#### `direnv`

Expand Down Expand Up @@ -89,19 +89,11 @@ Use the above command to rebuild the monorepo.

Before running tests: **follow the above instructions to get everything built.**

#### Running unit tests (typescript)

Run unit tests for all packages in parallel via:

```bash
pnpm test
```

To run unit tests for a specific package:
#### Running unit tests (solidity)

```bash
cd packages/package-to-test
pnpm test
cd packages/contracts-bedrock
just test
```

#### Running unit tests (Go)
Expand All @@ -125,7 +117,7 @@ To run `slither` locally, do:
```bash
cd packages/contracts-bedrock
pip3 install slither-analyzer
pnpm slither
just slither
```

## Labels
Expand Down
10 changes: 1 addition & 9 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -23,14 +23,6 @@ lint-go-fix: ## Lints Go code with specific linters and fixes reported issues
golangci-lint run -E goimports,sqlclosecheck,bodyclose,asciicheck,misspell,errorlint --timeout 5m -e "errors.As" -e "errors.Is" ./... --fix
.PHONY: lint-go-fix

build-ts: submodules ## Builds TypeScript components
if [ -f "$$NVM_DIR/nvm.sh" ]; then \
. $$NVM_DIR/nvm.sh && nvm use; \
fi
pnpm install:ci
pnpm build
.PHONY: build-ts

ci-builder: ## Builds the CI builder Docker image
docker build -t ci-builder -f ops/docker/ci-builder/Dockerfile .
.PHONY: ci-builder
Expand Down Expand Up @@ -207,7 +199,7 @@ test-unit: ## Runs unit tests for all components
make -C ./op-proposer test
make -C ./op-batcher test
make -C ./op-e2e test
pnpm test
(cd packages/contracts-bedrock && just test)
.PHONY: test-unit

# Remove the baseline-commit to generate a base reading & show all issues
Expand Down
44 changes: 44 additions & 0 deletions justfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
issues:
./ops/scripts/todo-checker.sh

lint-shellcheck:
find . -type f -name '*.sh' -not -path '*/node_modules/*' -not -path './packages/contracts-bedrock/lib/*' -not -path './packages/contracts-bedrock/kout*/*' -exec sh -c 'echo \"Checking $1\"; shellcheck \"$1\"' _ {} \\;

install-foundry:
curl -L https://foundry.paradigm.xyz | bash && just update-foundry

update-foundry:
bash ./ops/scripts/install-foundry.sh

check-foundry:
bash ./packages/contracts-bedrock/scripts/checks/check-foundry-install.sh

install-kontrol:
curl -L https://kframework.org/install | bash && just update-kontrol

update-kontrol:
kup install kontrol --version v$(jq -r .kontrol < versions.json)

install-abigen:
go install github.com/ethereum/go-ethereum/cmd/abigen@$(jq -r .abigen < versions.json)

print-abigen:
abigen --version | sed -e 's/[^0-9]/ /g' -e 's/^ *//g' -e 's/ *$//g' -e 's/ /./g' -e 's/^/v/'

check-abigen:
[[ $(just print-abigen) = $(cat versions.json | jq -r '.abigen') ]] && echo '✓ abigen versions match' || (echo '✗ abigen version mismatch. Run `just upgrade:abigen` to upgrade.' && exit 1)

upgrade-abigen:
jq '.abigen = $v' --arg v $(just print:abigen) <<<$(cat versions.json) > versions.json

install-slither:
pip3 install slither-analyzer==$(jq -r .slither < versions.json)

print-slither:
slither --version

check-slither:
[[ $(just print-slither) = $(jq -r .slither < versions.json) ]] && echo '✓ slither versions match' || (echo '✗ slither version mismatch. Run `just upgrade-slither` to upgrade.' && exit 1)

upgrade-slither:
jq '.slither = $v' --arg v $(just print-slither) <<<$(cat versions.json) > versions.json
2 changes: 1 addition & 1 deletion op-e2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,6 @@ make test-http

### Troubleshooting
If you encounter errors:
* ensure you have the latest version of foundry installed: `pnpm update:foundry`
* ensure you have the latest version of foundry installed: `just update-foundry`
* try deleting the `packages/contracts-bedrock/forge-artifacts` directory
* if the above step doesn't fix the error, try `pnpm clean`
14 changes: 3 additions & 11 deletions ops/docker/ci-builder/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -92,33 +92,25 @@ COPY --from=rust-build /root/.foundry/bin/anvil /usr/local/bin/anvil
COPY --from=rust-build /root/.cargo/bin/svm /usr/local/bin/svm
COPY --from=rust-build /root/.cargo/bin/just /usr/local/bin/just

COPY .nvmrc .nvmrc
COPY ./versions.json ./versions.json

ENV NODE_MAJOR=20

RUN /bin/sh -c set -eux; \
apt-get update; \
apt-get install -y --no-install-recommends bash curl openssh-client git build-essential ca-certificates jq gnupg binutils-mips-linux-gnu python3 python3-pip python3-setuptools; \
mkdir -p /etc/apt/keyrings; \
curl -fsSL https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor -o /etc/apt/keyrings/nodesource.gpg; \
echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_$NODE_MAJOR.x nodistro main" | tee /etc/apt/sources.list.d/nodesource.list; \
curl -fsSL https://download.docker.com/linux/debian/gpg | gpg --dearmor -o /etc/apt/keyrings/docker.gpg; \
chmod a+r /etc/apt/keyrings/docker.gpg; \
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/debian $(. /etc/os-release && echo "$VERSION_CODENAME") stable" | tee /etc/apt/sources.list.d/docker.list > /dev/null; \
apt-get update; \
apt-get install -y nodejs docker-ce-cli; \
apt-get install -y docker-ce-cli; \
ln -s /usr/local/go/bin/gofmt /usr/local/bin/gofmt; \
pip install capstone pyelftools; \
curl -fLSs https://raw.githubusercontent.com/CircleCI-Public/circleci-cli/master/install.sh | bash; \
apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false; \
rm -rf /var/lib/apt/lists/*; \
rm -rf /root/.cache/pip; \
rm -rf /root/.cache/npm;

RUN npm i -g pnpm && npm i -g yarn@1 && pnpm --version && yarn --version
rm -rf /root/.cache/pip;

RUN svm install 0.5.17 && \
RUN svm install 0.8.25 && \
svm install 0.8.15 && \
svm install 0.8.19

Expand Down
26 changes: 0 additions & 26 deletions package.json

This file was deleted.

3 changes: 1 addition & 2 deletions packages/contracts-bedrock/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,12 +51,11 @@ Enhancement suggestions are tracked as [GitHub issues](/issues).
The best place to begin contributing is by looking through the issues with the `good first issue` label. These are issues that are relatively easy to implement and are a great way to get familiar with the codebase.

Optimism's smart contracts are written in Solidity and we use [foundry](https://github.com/foundry-rs/foundry) as our development framework. To get started, you'll need to install several dependencies:
1. [pnpm](https://pnpm.io)
1. [just](https://github.com/casey/just)
1. Make sure to `just install`
1. [foundry](https://getfoundry.sh)
1. Foundry is built with [rust](https://www.rust-lang.org/tools/install), and this project uses a pinned version of foundry. Install the rust toolchain with `rustup`.
1. Make sure to install the version of foundry used by `ci-builder`, defined in the `versions.json` file in the root of this repo under the `foundry` key. Once you have `foundryup` installed, there is a helper to do this: `pnpm install:foundry`
1. Make sure to install the version of foundry used by `ci-builder`, defined in the `versions.json` file in the root of this repo under the `foundry` key. Once you have `foundryup` installed, there is a helper to do this: `just install-foundry`
1. [golang](https://golang.org/doc/install)
1. [python](https://www.python.org/downloads/)

Expand Down
2 changes: 1 addition & 1 deletion packages/contracts-bedrock/invariant-docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ This directory contains documentation for all defined invariant tests within `co

## Usage

To auto-generate documentation for invariant tests, run `pnpm autogen:invariant-docs`.
To auto-generate documentation for invariant tests, run `just autogen-invariant-docs`.

## Documentation Standard

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ VERSIONS_FILE="${MONOREPO_BASE}/versions.json"
if ! command -v forge &> /dev/null
then
# shellcheck disable=SC2006
echo "Is Foundry not installed? Consider installing via pnpm install:foundry" >&2
echo "Is Foundry not installed? Consider installing via just install-foundry" >&2
exit 1
fi

Expand All @@ -34,5 +34,5 @@ if [ "$INSTALLED_VERSION" = "$EXPECTED_VERSION" ]; then
else
echo "Mismatch between installed Foundry version ($INSTALLED_VERSION) and expected version ($EXPECTED_VERSION)."
echo "Your version of Foundry may either not be up to date, or it could be a later version."
echo "Running pnpm update:foundry will install the expected version."
echo "Running just update-foundry will install the expected version."
fi
Loading

0 comments on commit 770cbc6

Please sign in to comment.