Skip to content

Commit

Permalink
Make the smallest of edits to N/T README (MystenLabs/narwhal#134)
Browse files Browse the repository at this point in the history
* Make the smallest of edits to N/T README

* fix: replace references to master -> main

* chore: typo in README.md

Co-authored-by: François Garillot <[email protected]>
Co-authored-by: François Garillot <[email protected]>
  • Loading branch information
3 people authored Apr 18, 2022
1 parent bc64872 commit c4ce74d
Show file tree
Hide file tree
Showing 4 changed files with 21 additions and 21 deletions.
12 changes: 6 additions & 6 deletions narwhal/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,21 +4,21 @@
[![rustc](https://img.shields.io/badge/rustc-1.51+-blue?style=flat-square&logo=rust)](https://www.rust-lang.org)
[![license](https://img.shields.io/badge/license-Apache-blue.svg?style=flat-square)](LICENSE)

This repo provides an implementation of [Narwhal and Tusk](https://arxiv.org/pdf/2105.11827.pdf). The codebase has been designed to be small, efficient, and easy to benchmark and modify. It has not been designed to run in production but uses real cryptography ([dalek](https://doc.dalek.rs/ed25519_dalek)), networking ([tokio](https://docs.rs/tokio)), and storage ([rocksdb](https://docs.rs/rocksdb)).
This repo provides an implementation of [Narwhal and Tusk](https://arxiv.org/pdf/2105.11827.pdf), a [DAG](https://en.wikipedia.org/wiki/Directed_acyclic_graph)-based mempool and efficient [BFT](https://en.wikipedia.org/wiki/Byzantine_fault) consensus. The codebase has been designed to be small, efficient, and easy to benchmark and modify. It has not been designed to run in production but uses real cryptography ([dalek](https://doc.dalek.rs/ed25519_dalek)), networking ([tokio](https://docs.rs/tokio)), and storage ([rocksdb](https://docs.rs/rocksdb)).

## Quick Start
The core protocols are written in Rust, but all benchmarking scripts are written in Python and run with [Fabric](http://www.fabfile.org/).
To deploy and benchmark a testbed of 4 nodes on your local machine, clone the repo and install the python dependencies:
To deploy and benchmark a testbed of four nodes on your local machine, clone the repo and install the python dependencies:
```
$ git clone https://github.com/mystenlabs/narwhal.git
$ cd narwhal/benchmark
$ pip install -r requirements.txt
```
You also need to install Clang (required by rocksdb) and [tmux](https://linuxize.com/post/getting-started-with-tmux/#installing-tmux) (which runs all nodes and clients in the background). Finally, run a local benchmark using fabric:
You also need to install [Clang](https://clang.llvm.org/) (required by RocksDB) and [tmux](https://linuxize.com/post/getting-started-with-tmux/#installing-tmux) (which runs all nodes and clients in the background). Finally, run a local benchmark using fabric:
```
$ fab local
```
This command may take a long time the first time you run it (compiling rust code in `release` mode may be slow) and you can customize a number of benchmark parameters in `fabfile.py`. When the benchmark terminates, it displays a summary of the execution similarly to the one below.
This command may take a long time the first time you run it (compiling rust code in `release` mode may be slow), and you can customize a number of benchmark parameters in `fabfile.py`. When the benchmark terminates, it displays a summary of the execution similarly to the one below.
```
-----------------------------------------
SUMMARY:
Expand Down Expand Up @@ -52,9 +52,9 @@ This command may take a long time the first time you run it (compiling rust code
```

## Next Steps
The next step is to read the paper [Narwhal and Tusk: A DAG-based Mempool and Efficient BFT Consensus](https://arxiv.org/pdf/2105.11827.pdf). It is then recommended to have a look at the README files of the [worker](https://github.com/mystenlabs/narwhal/tree/master/worker) and [primary](https://github.com/mystenlabs/narwhal/tree/master/primary) crates. An additional resource to better understand the Tusk consensus protocol is the paper [All You Need is DAG](https://arxiv.org/abs/2102.08325) as it describes a similar protocol.
The next step is to read the paper [Narwhal and Tusk: A DAG-based Mempool and Efficient BFT Consensus](https://arxiv.org/pdf/2105.11827.pdf). It is then recommended to have a look at the README files of the [worker](https://github.com/mystenlabs/narwhal/tree/main/worker) and [primary](https://github.com/mystenlabs/narwhal/tree/main/primary) crates. An additional resource to better understand the Tusk consensus protocol is the paper [All You Need is DAG](https://arxiv.org/abs/2102.08325) as it describes a similar protocol.

The README file of the [benchmark folder](https://github.com/mystenlabs/narwhal/tree/master/benchmark) explains how to benchmark the codebase and read benchmarks' results. It also provides a step-by-step tutorial to run benchmarks on [Amazon Web Services (AWS)](https://aws.amazon.com) accross multiple data centers (WAN).
The README file of the [benchmark folder](https://github.com/mystenlabs/narwhal/tree/main/benchmark) explains how to benchmark the codebase and read benchmarks' results. It also provides a step-by-step tutorial to run benchmarks on [Amazon Web Services (AWS)](https://aws.amazon.com) across multiple data centers (WAN).

## License
This software is licensed as [Apache 2.0](LICENSE).
18 changes: 9 additions & 9 deletions narwhal/benchmark/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ This document explains how to benchmark the codebase and read benchmarks' result
When running benchmarks, the codebase is automatically compiled with the feature flag `benchmark`. This enables the node to print some special log entries that are then read by the python scripts and used to compute performance. These special log entries are clearly indicated with comments in the code: make sure to not alter them (otherwise the benchmark scripts will fail to interpret the logs).

### Parametrize the benchmark
After cloning the repo and [installing all dependencies](https://github.com/mystenlabs/narwhal#quick-start), you can use [Fabric](http://www.fabfile.org/) to run benchmarks on your local machine. Locate the task called `local` in the file [fabfile.py](https://github.com/mystenlabs/narwhal/blob/master/benchmark/fabfile.py):
After cloning the repo and [installing all dependencies](https://github.com/mystenlabs/narwhal#quick-start), you can use [Fabric](http://www.fabfile.org/) to run benchmarks on your local machine. Locate the task called `local` in the file [fabfile.py](https://github.com/mystenlabs/narwhal/blob/main/benchmark/fabfile.py):
```python
@task
def local(ctx):
Expand Down Expand Up @@ -104,7 +104,7 @@ $ ssh-keygen -f ~/.ssh/aws
```

### Step 3. Configure the testbed
The file [settings.json](https://github.com/mystenlabs/narwhal/blob/master/benchmark/settings.json) (located in [narwhal/benchmarks](https://github.com/mystenlabs/narwhal/blob/master/benchmark)) contains all the configuration parameters of the testbed to deploy. Its content looks as follows:
The file [settings.json](https://github.com/mystenlabs/narwhal/blob/main/benchmark/settings.json) (located in [narwhal/benchmarks](https://github.com/mystenlabs/narwhal/blob/main/benchmark)) contains all the configuration parameters of the testbed to deploy. Its content looks as follows:
```json
{
"key": {
Expand All @@ -115,7 +115,7 @@ The file [settings.json](https://github.com/mystenlabs/narwhal/blob/master/bench
"repo": {
"name": "narwhal",
"url": "https://github.com/mystenlabs/narwhal.git",
"branch": "master"
"branch": "main"
},
"instances": {
"type": "m5d.8xlarge",
Expand Down Expand Up @@ -144,7 +144,7 @@ The third block (`repo`) contains the information regarding the repository's nam
"repo": {
"name": "narwhal",
"url": "https://github.com/mystenlabs/narwhal.git",
"branch": "master"
"branch": "main"
},
```
Remember to update the `url` field to the name of your repo. Modifying the branch name is particularly useful when testing new functionalities without having to checkout the code locally.
Expand All @@ -159,12 +159,12 @@ The the last block (`instances`) specifies the [AWS instance type](https://aws.a
The instance type selects the hardware on which to deploy the testbed. For example, `m5d.8xlarge` instances come with 32 vCPUs (16 physical cores), 128 GB of RAM, and guarantee 10 Gbps of bandwidth. The python scripts will configure each instance with 300 GB of SSD hard drive. The `regions` field specifies the data centers to use. If you require more nodes than data centers, the python scripts will distribute the nodes as equally as possible amongst the data centers. All machines run a fresh install of Ubuntu Server 20.04.

### Step 4. Create a testbed
The AWS instances are orchestrated with [Fabric](http://www.fabfile.org) from the file [fabfile.py](https://github.com/mystenlabs/narwhal/blob/master/benchmark/fabfile.pyy) (located in [narwhal/benchmarks](https://github.com/mystenlabs/narwhal/blob/master/benchmark)); you can list all possible commands as follows:
The AWS instances are orchestrated with [Fabric](http://www.fabfile.org) from the file [fabfile.py](https://github.com/mystenlabs/narwhal/blob/main/benchmark/fabfile.pyy) (located in [narwhal/benchmarks](https://github.com/mystenlabs/narwhal/blob/main/benchmark)); you can list all possible commands as follows:
```
$ cd narwhal/benchmark
$ fab --list
```
The command `fab create` creates new AWS instances; open [fabfile.py](https://github.com/mystenlabs/narwhal/blob/master/benchmark/fabfile.py) and locate the `create` task:
The command `fab create` creates new AWS instances; open [fabfile.py](https://github.com/mystenlabs/narwhal/blob/main/benchmark/fabfile.py) and locate the `create` task:
```python
@task
def create(ctx, nodes=2):
Expand All @@ -189,13 +189,13 @@ This may take a long time as the command will first update all instances.
The commands `fab stop` and `fab start` respectively stop and start the testbed without destroying it (it is good practice to stop the testbed when not in use as AWS can be quite expensive); and `fab destroy` terminates all instances and destroys the testbed. Note that, depending on the instance types, AWS instances may take up to several minutes to fully start or stop. The command `fab info` displays a nice summary of all available machines and information to manually connect to them (for debug).

### Step 5. Run a benchmark
After setting up the testbed, running a benchmark on AWS is similar to running it locally (see [Run Local Benchmarks](https://github.com/mystenlabs/narwhal/tree/master/benchmark#local-benchmarks)). Locate the task `remote` in [fabfile.py](https://github.com/mystenlabs/narwhal/blob/master/benchmark/fabfile.py):
After setting up the testbed, running a benchmark on AWS is similar to running it locally (see [Run Local Benchmarks](https://github.com/mystenlabs/narwhal/tree/main/benchmark#local-benchmarks)). Locate the task `remote` in [fabfile.py](https://github.com/mystenlabs/narwhal/blob/main/benchmark/fabfile.py):
```python
@task
def remote(ctx):
...
```
The benchmark parameters are similar to [local benchmarks](https://github.com/mystenlabs/narwhal/tree/master/benchmark#local-benchmarks) but allow to specify the number of nodes and the input rate as arrays to automate multiple benchmarks with a single command. The parameter `runs` specifies the number of times to repeat each benchmark (to later compute the average and stdev of the results), and the parameter `collocate` specifies whether to collocate all the node's workers and the primary on the same machine. If `collocate` is set to `False`, the script will run one node per data center (AWS region), with its primary and each of its worker running on a dedicated instance.
The benchmark parameters are similar to [local benchmarks](https://github.com/mystenlabs/narwhal/tree/main/benchmark#local-benchmarks) but allow to specify the number of nodes and the input rate as arrays to automate multiple benchmarks with a single command. The parameter `runs` specifies the number of times to repeat each benchmark (to later compute the average and stdev of the results), and the parameter `collocate` specifies whether to collocate all the node's workers and the primary on the same machine. If `collocate` is set to `False`, the script will run one node per data center (AWS region), with its primary and each of its worker running on a dedicated instance.
```python
bench_params = {
'nodes': [10, 20, 30],
Expand All @@ -214,7 +214,7 @@ Once you specified both `bench_params` and `node_params` as desired, run:
```
$ fab remote
```
This command first updates all machines with the latest commit of the GitHub repo and branch specified in your file [settings.json](https://github.com/mystenlabs/narwhal/blob/master/benchmark/settings.json) (step 3); this ensures that benchmarks are always run with the latest version of the code. It then generates and uploads the configuration files to each machine, runs the benchmarks with the specified parameters, and downloads the logs. It finally parses the logs and prints the results into a folder called `results` (which is automatically created if it doesn't already exists). You can run `fab remote` multiple times without fearing to override previous results, the command either appends new results to a file containing existing ones or prints them in separate files. If anything goes wrong during a benchmark, you can always stop it by running `fab kill`.
This command first updates all machines with the latest commit of the GitHub repo and branch specified in your file [settings.json](https://github.com/mystenlabs/narwhal/blob/main/benchmark/settings.json) (step 3); this ensures that benchmarks are always run with the latest version of the code. It then generates and uploads the configuration files to each machine, runs the benchmarks with the specified parameters, and downloads the logs. It finally parses the logs and prints the results into a folder called `results` (which is automatically created if it doesn't already exists). You can run `fab remote` multiple times without fearing to override previous results, the command either appends new results to a file containing existing ones or prints them in separate files. If anything goes wrong during a benchmark, you can always stop it by running `fab kill`.

### Step 6. Plot the results
Once you have enough results, you can aggregate and plot them:
Expand Down
6 changes: 3 additions & 3 deletions narwhal/benchmark/data/latest/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Experimental Data

This folder contains some raw data and plots obtained running a geo-replicated benchmark on AWS as explained in the [benchmark's readme file](https://github.com/asonnino/narwhal/tree/master/benchmark#readme). The results are taken running the code tagged as [v0.2.0](https://github.com/asonnino/narwhal/tree/v0.2.0).
This folder contains some raw data and plots obtained running a geo-replicated benchmark on AWS as explained in the [benchmark's readme file](https://github.com/asonnino/narwhal/tree/main/benchmark#readme). The results are taken running the code tagged as [v0.2.0](https://github.com/asonnino/narwhal/tree/v0.2.0).

### Filename format
The filename format of raw data is the following:
Expand All @@ -18,7 +18,7 @@ where:
For instance, a file called `bench-0-50-1-True-100000-512.txt` indicates it contains results of a benchmark run with 50 nodes, 1 worker per node collocated on the same machine as the primary, 100K input rate, a transaction size of 512B, and 0 faulty nodes.

### Experimental step
The content of our [settings.json](https://github.com/asonnino/narwhal/blob/master/benchmark/settings.json) file looks as follows:
The content of our [settings.json](https://github.com/asonnino/narwhal/blob/main/benchmark/settings.json) file looks as follows:
```json
{
"key": {
Expand All @@ -37,7 +37,7 @@ The content of our [settings.json](https://github.com/asonnino/narwhal/blob/mast
}
}
```
We set the following `node_params` in our [fabfile](https://github.com/asonnino/narwhal/blob/master/benchmark/fabfile.py):
We set the following `node_params` in our [fabfile](https://github.com/asonnino/narwhal/blob/main/benchmark/fabfile.py):
```python
node_params = {
'header_size': 1_000, # bytes
Expand Down
6 changes: 3 additions & 3 deletions narwhal/benchmark/data/paper-data/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Experimental Data

This folder contains the raw data and plots used in the evaluation section of the paper [Narwhal and Tusk: A DAG-based Mempool and Efficient BFT Consensus](https://arxiv.org/pdf/2105.11827.pdf). The data are obtained running a geo-replicated benchmark on AWS as explained in the [benchmark's readme file](https://github.com/asonnino/narwhal/tree/master/benchmark#readme). The results are taken running the code tagged as [v0.2.0](https://github.com/asonnino/narwhal/tree/v0.2.0).
This folder contains the raw data and plots used in the evaluation section of the paper [Narwhal and Tusk: A DAG-based Mempool and Efficient BFT Consensus](https://arxiv.org/pdf/2105.11827.pdf). The data are obtained running a geo-replicated benchmark on AWS as explained in the [benchmark's readme file](https://github.com/asonnino/narwhal/tree/main/benchmark#readme). The results are taken running the code tagged as [v0.2.0](https://github.com/asonnino/narwhal/tree/v0.2.0).

### Filename format
The filename format of raw data is the following:
Expand All @@ -18,7 +18,7 @@ where:
For instance, a file called `bench-0-50-1-True-100000-512.txt` indicates it contains results of a benchmark run with 50 nodes, 1 worker per node collocated on the same machine as the primary, 100K input rate, a transaction size of 512B, and 0 faulty nodes.

### Experimental step
The content of our [settings.json](https://github.com/asonnino/narwhal/blob/master/benchmark/settings.json) file looks as follows:
The content of our [settings.json](https://github.com/asonnino/narwhal/blob/main/benchmark/settings.json) file looks as follows:
```json
{
"key": {
Expand All @@ -37,7 +37,7 @@ The content of our [settings.json](https://github.com/asonnino/narwhal/blob/mast
}
}
```
We set the following `node_params` in our [fabfile](https://github.com/asonnino/narwhal/blob/master/benchmark/fabfile.py):
We set the following `node_params` in our [fabfile](https://github.com/asonnino/narwhal/blob/main/benchmark/fabfile.py):
```python
node_params = {
'header_size': 1_000, # bytes
Expand Down

0 comments on commit c4ce74d

Please sign in to comment.