Skip to content

Commit

Permalink
Main Readme update (#2961)
Browse files Browse the repository at this point in the history 
* Update README.md

---------

Co-authored-by: Eyal Rundstein <[email protected]>
  • Loading branch information
@eyalrund
eyalrund and Eyal Rundstein authored Mar 22, 2023
1 parent fe49703 commit 5dbe330
Show file tree
Hide file tree
Showing 4 changed files with 41 additions and 288 deletions.
14 changes: 7 additions & 7 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ trigger the CI twice (on push and on PR), resulting in unecessary load on Github

Example:

``` shell
```shell
git remote add upstream https://github.com/vmware/concord-bft.git
git checkout -b my-new-feature master
git commit -a
Expand All @@ -32,7 +32,7 @@ git push origin my-new-feature

When your branch gets out of sync with the vmware/master branch, use the following to update:

``` shell
```shell
git checkout my-new-feature
git fetch -a
git pull --rebase upstream master
Expand All @@ -47,15 +47,15 @@ existing commits.
If your pull request contains a single commit or your changes are related to the most recent commit, you can simply
amend the commit.

``` shell
```shell
git add .
git commit --amend
git push --force-with-lease origin my-new-feature
```

If you need to squash changes into an earlier commit, you can use:

``` shell
```shell
git add .
git commit --fixup <commit>
git rebase -i --autosquash master
Expand All @@ -82,9 +82,9 @@ If you use emacs, consider installing

Our code style has the following exceptions that differ from the google C++ style guidelines.

* We use `camelCase` function and method names, rather than `PascalCase`.
* We allow, and encourage the use of exceptions.
* We allow and prefer `#pragma once` over include guards
* We use `camelCase` function and method names, rather than `PascalCase`.
* We allow, and encourage the use of exceptions.
* We allow and prefer `#pragma once` over include guards

## Reporting Bugs and Creating Issues

Expand Down
231 changes: 13 additions & 218 deletions README.md
View file
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
<img src="logoConcord.png"/>


# Concord-BFT: a Distributed Trust Infrastructure
# Concord-BFT: A Distributed Trust Framework

[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
[![clang-tidy](https://github.com/vmware/concord-bft/workflows/clang-tidy/badge.svg)](https://github.com/vmware/concord-bft/actions?query=workflow%3Aclang-tidy)
Expand All @@ -15,239 +14,35 @@

<!-- <img src="TODO.jpg" width="200" height="200" /> -->


## Overview

**Concord-bft** is a generic state machine replication library that can handle malicious (byzantine) replicas.

Its implementation is based on the algorithm described in the paper [SBFT: a Scalable Decentralized Trust Infrastructure for
Blockchains](https://arxiv.org/pdf/1804.01626.pdf).

It is designed to be used as a core building block for replicated distributed data stores, and is especially suited to serve as the basis of permissioned Blockchain systems.

For a real-life integration example, please take a look at [Project Concord](https://github.com/vmware/concord), a highly scalable and energy-efficient distributed trust infrastructure for consensus and smart contract execution.


## Releases

We abide by [semantic versioning](https://semver.org/). Public APIs can change
at any time until we reach version 1.0. We will, however, bump the minor version
when a backwards incompatible change is made.

[v0.5](https://github.com/vmware/concord-bft/releases/tag/v0.5)

## Install and Build (Ubuntu Linux 18.04)

Concord-BFT supports two kinds of builds: native and docker.

The docker build is **strongly recommended**.

### Docker

* Install the latest docker.
* Optional: [configure docker as non-root user](https://docs.docker.com/engine/install/linux-postinstall/#manage-docker-as-a-non-root-user).
* Build:
```sh
cd concord-bft
make
make test
```
Run `make help` to see more commands.

Note:
* The output binaries are stored in the host's `concord-bft/build`.
* `Makefile` is configurable.
For example, if you want to use another compiler you may pass it to the `make`:
```
make CONCORD_BFT_CONTAINER_CXX=g++ \
CONCORD_BFT_CONTAINER_CC=gcc \
build
```

Other build options, including passthrough options for CMake, are defined in the Makefile and prefixed with `CONCORD_BFT_`. Variables that are capable of being overridden on the commandline are set with the Make conditional operator `?=` and are at the beginning of [Makefile](Makefile). Please check that file for options.

#### Select comm module
One option that is worth calling out explicitly is the communication (transport) library. Transport defaults to TLS and can be configured explicitly by setting the `CONCORD_BFT_CMAKE_TRANSPORT` flag. The flag defaults to **TLS**, but also supports **UDP** and **TCP**. These can be useful because the use of pinned certificates for TLS requires an out of band setup.

See [create_tls_certs.sh](scripts/linux/create_tls_certs.sh) for an example. This script is used in apollo tests. For production usage, an out of band deployment for each replica must be used to avoid revealing private keys to each replica.

### Native

```sh
git clone https://github.com/vmware/concord-bft
cd concord-bft
sudo ./install_deps_release.sh # Installs all dependencies and 3rd parties
mkdir build
cd build
cmake ..
make
sudo make test
```

By default the build chooses the active compiler on the native build platform. In order to force compilation by clang you can use the following command.
```sh
CC=clang CXX=clang++ cmake ..
```

In order to turn on or off various options, you need to change your cmake configuration. This is
done by passing arguments to cmake with a `-D` prefix: e.g. `cmake -DBUILD_TESTING=OFF`. Note that
make must be run afterwards to build according to the configuration. Please see [CMakeLists.txt](CMakeLists.txt) for configurable options.

#### Select comm module
One option that is worth calling out explicitly is the communication (transport) library.

We support both UDP and TCP communication. UDP is the default. In order to
enable TCP communication, build with `-DBUILD_COMM_TCP_PLAIN=TRUE` in the cmake
instructions shown above. If set, the test client will run using TCP. If you
wish to use TCP in your application, you need to build the TCP module as
mentioned above and then create the communication object using CommFactory and
passing PlainTcpConfig object to it.

We also support TCP over TLS communication. To enable it, change the
`BUILD_COMM_TCP_TLS` flag to `TRUE` in the main CMakeLists.txt file. When
running simpleTest using the testReplicasAndClient.sh - there is no need to create TLS certificates manually. The script will use the `create_tls_certs.sh` (located under the scripts/linux folder) to create certificates. The latter can be used to create TLS files for any number of replicas, e.g. when extending existing tests.

As we used pinned certificates for TLS, the user will have to manually provide these. THey can use the [create_tls_certs.sh](scripts/linux/create_tls_certs.sh) script as an example.


### C++ Linter

The C++ code is statically checked by `clang-tidy` as part of the [CI](https://github.com/vmware/concord-bft/actions?query=workflow%3Aclang-tidy).
<br>To check code before submitting PR, please run `make tidy-check`.

[Detailed information about clang-tidy checks](https://clang.llvm.org/extra/clang-tidy/checks/list.html).

BFT-based systems require substantial communication between nodes and, thus, don’t scale well. Project Concord-bft solves this problem by simplifying and streamlining communication between nodes, enabling greater scalability while increasing overall network throughput.

#### (Optional) Python client
Project Concord’s BFT engine obtains significant scaling improvements via three major advances:

The python client is required for running tests. If you do not want to install python, you can
configure the build of concord-bft by running `cmake -DBUILD_TESTING=OFF ..` from the `build`
directory for native builds, and `CONCORD_BFT_CMAKE_BUILD_TESTING=TRUE make` for docker builds.
1. It uses a linear communication consensus protocol while many other BFT consensus protocols (including PBFT) require quadratic communication
2. It exploits optimism to provide a common case fast-path execution
3. It uses modern cryptographic algorithms (BLS threshold signatures)

The python client requires python3(>= 3.5) and trio, which is installed via pip.

python3 -m pip install --upgrade trio


#### Adding a new dependency or tool

The CI builds and runs tests in a docker container. To add a new dependency or tool, follow the steps below:

* Rebase against master
* In order to add/remove dependencies update the file
[install_deps_release.sh](https://github.com/vmware/concord-bft/blob/master/install_deps_release.sh)
* Build new release/debug images: `make build-docker-images`
* Check images current version in
[Makefile](https://github.com/vmware/concord-bft/blob/master/Makefile#L5)
and
[Makefile](https://github.com/vmware/concord-bft/blob/master/Makefile#L10)
* Tag the new images (at least one):
* For release: `docker tag concord-bft:latest concordbft/concord-bft:<version>`
* For debug: `docker tag concord-bft-debug:latest concordbft/concord-bft-debug:<version>`
<br>where version is `current version + 1`.
* Update the version in the Makefile
* Make sure that Concord-BFT is built and tests pass with the new images: `RUN_WITH_DEBUG_IMAGE=TRUE make
clean build test`
* Ask one of the maintainers for a temporary write permission to Docker Hub
repository(you need to have a [Docker ID](https://docs.docker.com/docker-id/))
* Push the images (at least one):
* For release: `docker push concordbft/concord-bft:<version>`
* For debug: `docker push concordbft/concord-bft-debug:<version>`
* Create a PR for the update:
* The PR must contain only changes related to the updates in the image
* PR's summary has to be similar to `Docker update to version release=<new version> debug=<new version>`
* PR's message has to list the changes made in the image content and
preferably the reason
* Submit the PR

Important notes:
1. Adding dependencies or tools directly to the `Dockerfile` is strongly not recommended because it breaks the native build support.
2. If any tools are installed during the build but not needed for the actual compilation/debugging/test execution(for example, `git`), please remove them(`Dockerfile` is an example). The reason is that the image is supposed to be as tiny as possible.


## Apollo testing framework


The Apollo framework provides utilities and advanced testing scenarios for validating
Concord BFT's correctness properties, regardless of the running application/execution engine.
For the purposes of system testing, we have implemented a "Simple Key-Value Blockchain" (SKVBC)
test application which runs on top of the Concord BFT consensus engine.
<br>

Apollo enables running all test suites (without modification) against any supported BFT network
configuration (in terms of <i>n</i>, <i>f</i>, <i>c</i> and other parameters).
<br>

Various crash or byzantine failure scenarios are also covered
(including faulty replicas and/or network partitioning).
<br>

Apollo test suites run regularly as part of Concord BFT's continuous integration pipeline.

Please find more details about the Apollo framework [here](tests/apollo/README.md)

## Run examples


### Simple test application (4 replicas and 1 client on a single machine)

Tests are compiled into in the build directory and can be run from anywhere as
long as they aren't moved.

Run the following from the top level concord-bft directory:

./build/tests/simpleTest/scripts/testReplicasAndClient.sh

### Using simple test application via Python script

You can use the simpleTest.py script to run various configurations via a simple
command line interface.
Please find more information [here](./tests/simpleTest/README.md)

### Example application
This example demo application shows some capabilities of the Concord-BFT consensus-based byzantine fault-tolerant state machine replication library.
For Concord-BFT users who are interested in learning more about Concord-BFT and its uses, this application offers a demonstration and instruction.
Overall, any blockchain application based on concord-bft consensus may be created using this example application.
<br>

Use the [test_example.sh](example/scripts/test_example.sh) script to run the example application. This script is also used to perform this demo via the command line interface with different configurations.
<br>

Please see [here](example/README.md) for more information about the example/demo application.

## Directory Structure


- [bftengine](./bftengine): concord-bft codebase
- [include](./bftengine/include): external interfaces of concord-bft (to be used by client applications)
- [src](./bftengine/src): internal implementation of concord-bft
- [tests](./bftengine/tests): tests and usage examples
- [threshsign](./threshsign): crypto library that supports digital threshold signatures
- [include](./threshsign/include): external interfaces of threshsign (to be used by client applications)
- [src](./threshsign/src): internal implementation of threshsign
- [tests](./threshsign/tests): tests and usage examples
- [scripts](./scripts): build scripts
- [tests](./tests): BFT engine system tests
Its implementation is based on the algorithm described in the paper [SBFT: a Scalable Decentralized Trust Infrastructure for
Blockchains](https://arxiv.org/pdf/1804.01626.pdf).

## Contributing
It is designed to be used as a core building block for replicated distributed data stores, and is especially suited to serve as the basis of permissioned Blockchain systems.

Start with example usage [here](https://github.com/vmware/concord-bft/tree/master/examples).

The concord-bft project team welcomes contributions from the community. If you wish to contribute code and you have not
signed our contributor license agreement (CLA), our bot will update the issue when you open a Pull Request. For any
questions about the CLA process, please refer to our [FAQ](https://cla.vmware.com/faq). For more detailed information,
refer to [CONTRIBUTING.md](CONTRIBUTING.md).
## Documentation

## Notes
The library calls `std::terminate()` when it cannot continue in a safe manner.
In that way, users can install a handler that does something different than just calling `std::abort()`.
See the github [wiki](https://github.com/vmware/concord-bft/wiki) for detailed explanation.

## Community


[Concord-BFT Slack](https://concordbft.slack.com/).

Request a Slack invitation via <[email protected]>.

## License

concord-bft is available under the [Apache 2 license](LICENSE).
Concord-bft is available under the [Apache 2 license](LICENSE).
4 changes: 3 additions & 1 deletion docs/design.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ explanatory. Important structures will be mentioned explicitly below.
All user interfaces live in the `bftEngine` namespace.

# Clients

A default client,
[SimpleClient](../bftengine/include/bftengine/SimpleClient.hpp), is included
with concord-bft. It is just one manner of implementing a client, and is
Expand All @@ -32,6 +33,7 @@ Python](../util/pyclient/bft_client.py) that we use for testing that does just
that.

## Client Messages

Client request and reply messages are prefixed with a `ClientRequestMsgHeader` or
`ClientReplyMsgHeader` respectively. Each header, like all other messages in
concord-bft is implemented as a packed C struct, meaning that there are no padding
Expand Down Expand Up @@ -111,7 +113,7 @@ committed. This is the hook that allows arbitrary SMR implemented in concord-bft
to result in execution of application specific operations at all replicas in a
total order.

# ReplicaConfig
## ReplicaConfig
ReplicaConfig contains most configurable attributes of concord-bft and should be
created by the application and passed into `Replica::createReplica(...)`.
Note that other configurable attributes are set as part of the communication
Expand Down
Loading

0 comments on commit 5dbe330

Please sign in to comment.