Network

Monorepo containing all the main components of Streamr Network.

Table of Contents

• Packages
• NPM scripts
• Release

Packages

User-Facing

• broker (streamr-broker)
• client (streamr-client)
• cli-tools (@streamr/cli-tools)

Internal

• network-node (@streamr/network-node)
• network-tracker (@streamr/network-tracker)
• protocol (@streamr/protocol)
• utils (@streamr/utils)
• test-utils (@streamr/test-utils)

NPM scripts

Node.js 16.13.x is the minimum required version. Node.js 18.12.x, NPM 8.x and later versions are recommended.

The monorepo is managed using npm workspaces.

Installation on an Apple Silicon Mac requires additional steps, see install-on-apple-silicon.md.

Important: Do not use npm ci or npm install directly in the sub-package directories.

Bootstrap all sub-packages

The go-to command for most use cases.

To install all required dependencies and build all sub-packages (linking sub-packages together as needed):

# from top level
npm run bootstrap

Bootstrap a single sub-package

To install the required dependencies and build a specific sub-package:

# from top level
npm run bootstrap-pkg --package=$PACKAGE_NAME

Install dependencies only

To only install required dependencies and link sub-packages together (and skip build phase):

# from top level
npm ci

Build

To build all sub-packages:

# from top level
npm run build

Build a sub-package

To build a specific sub-package:

# from top level
npm run build --workspace=$PACKAGE_NAME

Clear caches and built files

To clear all caches and remove the dist directory from each sub-package:

# from top level
npm run clean-dist

Clean all

To removes all caches, built files, and node_modules of each sub-package, and the top-level node_modules:

# from top level
npm run clean

Install git hooks

To install git hooks (e.g. Husky for conventional commit validation):

npm run install-git-hooks

Add a dependency into a sub-package

Manually add the entry to the package.json of the sub-package and run npm run bootstrap-pkg $PACKAGE_NAME.

Alternatively, run:

npm install some-dependency --workspace=$PACKAGE_NAME

List active versions & symlinks

Check which sub-packages are currently being symlinked.

# from top level
npm run versions

This lists sub-packages & their versions on the left, linked sub-packages are columns. If the package on the left links to the package in the column, it shows a checkmark & the semver range, otherwise it shows the mismatched semver range and prints a warning at the end. It prints the version ranges so you can double-check that they're formatted as you expect e.g. ^X.Y.Z vs X.Y.Z

Release

utils, test-utils, protocol, network-tracker, network-node, client, cli-tools

All the above packages should be released at the same time.

1. git checkout main
2. git pull
3. ./update-versions.sh <SEMVER> E.g. ./update-versions.sh 7.1.1
4. npm run clean && npm install && npm run build && npm run versions
5. Look at the output of the above and ensure all versions are linked properly (i.e. no yellow or red markers)
6. Update client and cli-tool CHANGELOG.md
7. If releasing a major / minor version, update API docs link in packages/client/README.md.
8. Add relevant files to git staging
9. git commit -m "release(client, cli-tools): vX.Y.Z"
10. git tag client/vX.Y.Z
11. git tag cli-tools/vX.Y.Z
12. Push to main git push origin
13. Push to tag git push origin client/vX.Y.Z
14. Push to tag git push origin cli-tools/vX.Y.Z
15. At this point we are to do the actual release
16. Clean and rebuild project with npm run clean && npm run bootstrap
17. Then we do actual publishing of packages with ./release.sh <NPM_TAG>. Use argument beta if publishing a beta version. Use latest instead when publishing a stable version.
18. Update client docs if major or minor change:

# Generate & upload API docs (if a major/minor version update)
cd packages/client
npm run docs
aws s3 cp ./docs s3://api-docs.streamr.network/client/vX.Y --recursive --profile streamr-api-docs-upload

broker

Broker is released independently of other packages because it follows its own versioning for the time being.

git checkout main
cd packages/broker
npm version <SEMVER_OPTION>
git add package.json package-lock.json
git commit -m "release(broker): vX.Y.Z"
git tag broker/vX.Y.Z
git push --atomic origin main broker/vX.Y.Z

npm run build
npm publish

Docker release

After pushing the broker tag, GitHub Actions will build and publish the Docker image automatically if tests pass.

Tag latest

GitHub Actions will not update the latest tag. This must be done manually. Keep in mind that latest should always refer to the latest stable version.

To update latest do the following.

1. Find out the sha256 digests of both the amd64 and arm64 builds for a vX.Y.Z tag. This can be done via command-line docker buildx imagetools inspect streamr/broker-node:vX.Y.Z or you can check this from docker hub website under https://hub.docker.com/r/streamr/broker-node/tags.
2. Then we shall create the manifest by running the below. Remember to replace <SHA-AMD64> and <SHA-ARM64> with real values.

docker manifest create streamr/broker-node:latest \
    --amend streamr/broker-node@sha256:<SHA-AMD64> \
    --amend streamr/broker-node@sha256:<SHA-ARM64>

3. Then we publish the manifest with

docker manifest push streamr/broker-node:latest

4. Then we are ready. It would be wise to double-check this by checking https://hub.docker.com/r/streamr/broker-node/tags.