Skip to content

Commit

Permalink
Merge pull request autonomys#385 from subspace/cleanup
Browse files Browse the repository at this point in the history
Cleanup for docs and code owners
nazar-pc authored Apr 24, 2022
2 parents 98f7e25 + e84aa2f commit 004691c
Showing 7 changed files with 57 additions and 464 deletions.
21 changes: 11 additions & 10 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
@@ -2,13 +2,14 @@
# below ones takes precedence over the upper ones

# Global owners
* @nazar-pc @rg3l3dr

# Scripts
/scripts/ @ImmaZoni @nazar-pc @rg3l3dr

# Protocol
/crates/ @liuchengxu @nazar-pc @rg3l3dr

# Farmer
/crates/subspace-farmer/ @ozgunozerk @liuchengxu @nazar-pc @rg3l3dr
* @nazar-pc @rg3l3dr

/crates @liuchengxu @nazar-pc @rg3l3dr
/crates/pallet-* @liuchengxu @nazar-pc @rg3l3dr @vedhavyas
/crates/sp-* @liuchengxu @nazar-pc @rg3l3dr @vedhavyas
/crates/subspace-archiving @liuchengxu @i1i1 @nazar-pc @rg3l3dr
/crates/subspace-farmer @i1i1 @nazar-pc @rg3l3dr
/crates/subspace-networking @nazar-pc @rg3l3dr @shamil-gadelshin
/crates/subspace-runtime* @liuchengxu @nazar-pc @rg3l3dr @vedhavyas
/crates/subspace-solving @i1i1 @liuchengxu @nazar-pc @rg3l3dr
/crates/substrate @nazar-pc @rg3l3dr
41 changes: 22 additions & 19 deletions crates/subspace-farmer/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
<div align="center">
<h1><code>subspace-farmer</code></h1>
<strong>Farmer for the <a href="https://subspace.network/">Subspace Network Blockchain</a></strong>
</div>
# Subspace Node

Reference implementation of Subspace Farmer for Subspace Network Blockchain.

## Overview
**Notes:** The code is un-audited and not production ready, use it at your own risk.
@@ -18,46 +17,50 @@ Plotting time is roughly linear with respect to number of cores and clock speed

### Storage Overhead

In addition, the plot, a small Binary Search Tree (BST) is also stored on disk using RocksDB. This adds roughly 1% storage overhead. So creating a 1GB plot will actually consume about 1.01 GB of storage.
In addition to the plot a small Binary Search Tree (BST) is also stored on disk using RocksDB, which has roughly 1% of the storage size.
Due to current implementation two of such databases might be stored at once, though this will improve in the future.
There are also some supplementary database mappings.

## Run with Docker
The simplest way to use subspace-farmer is to use container image:
```bash
docker volume create subspace-farmer
docker run --rm -it --mount source=subspace-farmer,target=/var/subspace subspacelabs/subspace-farmer --help
```
So creating a 1GB plot should actually consume about 1.03 GB of storage.
Plot size parameter specified in farming command accounts for this overhead, so you don't need to worry about implementation details.

`subspace-farmer` is the volume where plot and identity will be stored, it only needs to be created once.
## Running

## Install and Run Manually
Instead of Docker you can also install subspace-farmer natively by compiling it using cargo.
It is recommended to follow general farming instructions that explain how to run both farmer and node together.

## Build from source

Rust toolchain is expected to be installed for anything in this repository to compile, but there are some extra dependencies for farmer specifically.

RocksDB on Linux needs LLVM/Clang:
```bash
sudo apt-get install llvm clang
```

Then install the framer using Cargo:
Then build the farmer using Cargo:
```
cargo install subspace-farmer
cargo build --profile production subspace-farmer
target/production/subspace-farmer --version
```

## Usage
Commands here assume you installed native binary, but you can also easily adapt them to using with Docker.

Use `--help` to find out all available commands and their options:
```
subspace-farmer --help
target/production/subspace-farmer --help
```

### Start the farmer
```
subspace-farmer farm
target/production/subspace-farmer farm --reward-address st... --plot-size 100G
```

`st...` should be replaced with the reward address taken from Polkadot.js wallet (or similar) and `100G` replaced with desired plot size.

This will connect to local node and will try to solve on every slot notification, while also plotting all existing and new history of the blockchain in parallel.

*NOTE: You need to have a subspace-client node running before starting farmer, otherwise it will not be able to start*
*NOTE: You need to have a `subspace-node` running before starting farmer, otherwise it will not be able to start*

By default, plots are written to the OS-specific users local data directory.

231 changes: 24 additions & 207 deletions crates/subspace-node/README.md
Original file line number Diff line number Diff line change
@@ -1,237 +1,54 @@
<div align="center">
# Subspace Node

<h1><code>subspace-node</code></h1>

<strong>A Subspace Network Blockchain node.</strong>

</div>

# Overview

This repo is an implementation of Substrate consensus on Substrate framework.

# Subspace node
Reference Node implementation for Subspace Network Blockchain using [Substrate](https://docs.substrate.io/) framework.

## Getting Started

Follow these steps to get started with the Subspace Node :hammer_and_wrench:

Note that this repo is for running a Subspace node. In order to run a full node which participates in consensus and produces blocks you must also run a subspace-farmer.
## Running

### Run with Docker
It is recommended to follow general farming instructions that explain how to run both farmer and node together.

**Note:** These instructions assume you run the farmer in one terminal and the client in a second terminal.
## Build from source

First, install [Docker](https://docs.docker.com/get-docker/).
Rust toolchain is expected to be installed for anything in this repository to compile, but there are some extra dependencies for farmer specifically.

#### Run the Client (Terminal 1)

Create virtual network, pull the latest image and start a single node development chain:
RocksDB on Linux needs LLVM/Clang:
```bash
docker network create subspace
docker pull subspacelabs/subspace-node
docker run --rm --init -it \
--net subspace \
--name subspace-node \
--publish 127.0.0.1:30333:30333 \
--publish 127.0.0.1:9944:9944 \
--publish 127.0.0.1:9933:9933 \
subspacelabs/subspace-node \
--dev \
--tmp \
--ws-external \
--node-key 0000000000000000000000000000000000000000000000000000000000000001
```

#### Run the Farmer (Terminal 2)

Once node is running, create volume for plot, pull the latest image and connect farmer to the node by running following in a separate terminal:
```bash
docker volume create subspace-farmer
docker pull subspacelabs/subspace-farmer
docker run --rm --init -it \
--net subspace \
--name subspace-farmer \
--mount source=subspace-farmer,target=/var/subspace \
subspacelabs/subspace-farmer \
farm \
--node-rpc-url ws://subspace-node:9944
sudo apt-get install llvm clang
```

Now you should see block production in the first terminal where node is running.

#### Running Full Client

We can now run another full client and sync the chain from the client we started earlier:
Then build the farmer using Cargo:
```
BOOTSTRAP_CLIENT_IP=$(docker inspect -f "{{.NetworkSettings.Networks.subspace.IPAddress}}" subspace-node)
docker run --rm --init -it \
--net subspace \
--name subspace-node-full \
subspacelabs/subspace-node \
--dev \
--tmp \
--ws-external \
--bootnodes /ip4/$BOOTSTRAP_CLIENT_IP/tcp/30333/p2p/12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp
cargo build --profile production subspace-node
target/production/subspace-node --version
```

#### Running Light Client
#### Start the node

We can also run light client and sync the chain from the client we started earlier:
```
BOOTSTRAP_CLIENT_IP=$(docker inspect -f "{{.NetworkSettings.Networks.subspace.IPAddress}}" subspace-node)
docker run --rm --init -it \
--net subspace \
--name subspace-node-light \
subspacelabs/subspace-node \
Start a single node development chain:
```bash
target/production/subspace-node \
--dev \
--tmp \
--light \
--ws-external \
--bootnodes /ip4/$BOOTSTRAP_CLIENT_IP/tcp/30333/p2p/12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp
```

#### Run more nodes on the test network

If above setup is not enough, you can use `run-node-farmer-pair.sh` script to run more full nodes on the network, each with own farmer.

Usage is simple:
```
./run-node-farmer-pair.sh test
```

Where `test` is the name of the pair. You can create as many pairs as needed, they will all join the same test network.

Use `Ctrl+C` to stop the pair, everything will be stopped and cleaned up automatically.

### Run In Development Mode

#### Install Rust

Install Rust toolchain with [rustup.rs](https://rustup.rs/).

If you didn't re-login yet, make sure configure your shell in the meantime:
```bash
source ~/.cargo/env
```

Install nightly toolchain and wasm32 target for it:
```bash
rustup toolchain install nightly
rustup target add wasm32-unknown-unknown --toolchain nightly
```

#### Install Dependencies

On Linux, RocksDB requires Clang:
```bash
sudo apt-get install llvm clang
```

#### Install and Run Node

This will run a subspace-node in one terminal and a subspace-farmer farming in a second terminal.
The node will send slot notification challenges to the farmer.
If the farmer finds a valid solution it will reply, and the node will produce a new block.

```bash
# Get source code
git clone https://github.com/subspace/subspace.git
cd subspace

# Build and run Node (first terminal)
cargo run --bin subspace-node -- --dev --tmp

# wait for the client to start before continuing...

# Run Farmer (second terminal)
cargo run --bin subspace-farmer -- farm
--node-key 0000000000000000000000000000000000000000000000000000000000000001
```

### Test equivocation behavior
1. Run bootstrap client node with farmer according to instructions in "Run with Docker" section above
2. Start the first full client node and farmer with the same identity as the bootstrap client node:
1. In one terminal run full client:
```bash
BOOTSTRAP_CLIENT_IP=$(docker inspect -f "{{.NetworkSettings.Networks.subspace.IPAddress}}" subspace-node)
docker run --rm --init -it \
--net subspace \
--name subspace-node-full-1 \
subspacelabs/subspace-node \
--dev \
--tmp \
--ws-external \
--bootnodes /ip4/$BOOTSTRAP_CLIENT_IP/tcp/30333/p2p/12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp
```
2. In another terminal plot with the same identity:
```bash
docker volume create subspace-farmer-1
docker run --rm -it \
--entrypoint=/bin/cp \
--mount source=subspace-farmer,target=/var/subspace-src \
--mount source=subspace-farmer-1,target=/var/subspace \
subspacelabs/subspace-farmer cp /var/subspace-src/identity.bin /var/subspace/identity.bin
docker run --rm -it \
--name subspace-farmer-1 \
--mount source=subspace-farmer-1,target=/var/subspace \
subspacelabs/subspace-farmer plot 256000 subspace
```
3. And start farming while being connected to the full client:
```bash
docker run --rm --init -it \
--net subspace \
--name subspace-farmer-1 \
--mount source=subspace-farmer-1,target=/var/subspace \
subspacelabs/subspace-farmer \
farm \
--node-rpc-url ws://subspace-node-full-1:9944
```
3. Repeat 2. with `-1` replaced with `-2` everywhere in order to obtain one more pair of client and farmer
4. Observe following messages in logs similar to these, also block production will stop:
```
Slot author Public(X (Y...)) is equivocating at slot Z with headers W and A
Submitted Subspace equivocation report.
Submitted equivocation report for author Public(X (Y...))
Ignoring solution for slot X provided by farmer in block list: Y
```
### Run Tests
#### Start full node

You can now run another full node and sync the chain from the node started earlier:
```bash
cargo test
target/production/subspace-node \
--dev \
--ws-external \
--bootnodes /ip4/127.0.0.1/tcp/30333/p2p/12D3KooWEyoppNCUx8Yx66oV9fJnriXwCcXwDDUA2kj6vnc6iDEp
```

### Embedded Docs

Once the project has been built, the following command can be used to explore all parameters and
subcommands:
```bash
cargo run --bin subspace-node -- --help
```
## Run
The provided `cargo run` command will launch a temporary node and its state will be discarded after
you terminate the process. After the project has been built, there are other ways to launch the
node.
### Single-Node Development Chain
This command will start the single-node development chain with persistent state:
```bash
cargo run --bin subspace-node -- --dev
```
Purge the development chain's state:
```bash
cargo run --bin subspace-node -- purge-chain --dev
```
Start the development chain with detailed logging:
Once the project has been built, the following command can be used to explore all parameters and subcommands:

```bash
RUST_BACKTRACE=1 cargo run --bin subspace-node -- -ldebug --dev
target/production/subspace-node --help
```
41 changes: 0 additions & 41 deletions scripts/docker-compose.yml

This file was deleted.

24 changes: 0 additions & 24 deletions scripts/run-node-farmer-pair.sh

This file was deleted.

81 changes: 0 additions & 81 deletions scripts/start-farmer.sh

This file was deleted.

82 changes: 0 additions & 82 deletions scripts/start-node.sh

This file was deleted.

0 comments on commit 004691c

Please sign in to comment.