Skip to content

Commit

Permalink
Add Slp Graph Search feature (gcash#451)
Browse files Browse the repository at this point in the history
* add slp graph search core

* gs server updates, only mempool handling remains

* add mempool access to graph search

* update sample-bchd.conf

* update generated pb for web proxy

* update generated pb.go and swagger

* Use multiple goroutines to load the graph search db

I just hardcoded maxGoroutines  at 4 as seems the db loading process doesn't get much better with more goroutines.  Loading time with 1M items is a little over 1 minute.  Before this it took around 4-5 minutes.

There are a couple of optimizations noted (see TODOs)

* clean up debug slp debug statements

* clean up debug slp debug statements

* improve loading experience

* use cpu count * 3 for max goroutines
* log loading status at info log level

* clean up logging and notation

* add gs status to GetBlockchainInfo

* return proper error for GetGraphSearchDb

during the loading period the db isn't available so
we'll return this as an error so its clear why its not
available

* refactor into standalone package

* fix lock

* topo sort new block txns

* remove debug lines

* clean up mempool/block handling

With GS added some renaming was done since
both slp index and gs are sharing the same event
handler.  Also some of the logging messages was
cleaned up.

* fix error handling in addTxn

* refactor event handling

* improved debug logging

* remove stray fmt.Println()

* more refactoring and containment of gs

* use original prop names

* fix state changes

* nit: use "hash" vs "txid" in naming validity cache

* update bchutil version

* update goslp version

* grpc server address property updates

* for grpc endpoints GetAddressTransactions and GetAddressUnspentOutputs
convert slp to cash address
* include slp address property in input metadata for SubscribeTransactoins
endpoints on inputs

* remove address prefix in response

* add check for slp parsing error

* update generated files to include graphsearch

* remove bad type reference

* Revert "remove GetBip44HdAddress grpc endpoint"

This reverts commit f3cfa5a.

* fixes for including Bip44 endpoint

* use cache in ConnectBlock

* use RLock as much as possible

* follow-up merge commit

* nits

* follow up merge

* refactored gs parts

* fix performance issues from locking

* using atomic lock to read gs db state seemed to degrade performance significantly,
switched to RLock before commiting to a Lock

* reduced the number of times any kind of lock is being held

* impr debug logging

* set gs db ready flag

* separate flag checks for gs

* remove GetBip44HdAddress

* impr err resp

* peer sync tweaks from btcd

* Revert "peer sync tweaks from btcd"

This reverts commit df5730b.

* run protoc after merge

* enable slpgraphsearch for bchrpc regtests

* put gs AddTx update in goroutine

* fix pb generated files (merge conflicts)

* improve handling the removal of slp mempool items

* fixes error of having RemoveMempoolSlpTxs inside a for loop in
ConnectBlock

* make RemoveMempoolSlpTxs a private method and remove it from the
grpc server block event

* ignore non-contributing parents

* fix generated pb after merge

* update bindata for a clean merge

* unit test updates:
- update proxy test url
- update regtest npm pkg versions

Co-authored-by: Ekliptor <[email protected]>
  • Loading branch information
jcramer and Ekliptor authored May 12, 2021
1 parent d969fad commit 06690af
Show file tree
Hide file tree
Showing 20 changed files with 2,509 additions and 1,238 deletions.
19 changes: 19 additions & 0 deletions bchrpc/bchrpc.proto
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,11 @@ service bchrpc {
// **Requires SlpIndex**
rpc GetSlpTrustedValidation(GetSlpTrustedValidationRequest) returns (GetSlpTrustedValidationResponse) {}

// GraphSearch returns all the transactions needed for a client to validate an SLP graph
//
// **Requires SlpIndex and SlpGraphSearch**
rpc GetSlpGraphSearch (GetSlpGraphSearchRequest) returns (GetSlpGraphSearchResponse) {}

// CheckSlpTransaction checks the validity of a supposed slp transaction before it is broadcasted.
rpc CheckSlpTransaction(CheckSlpTransactionRequest) returns (CheckSlpTransactionResponse) {}

Expand Down Expand Up @@ -202,6 +207,8 @@ message GetBlockchainInfoResponse {
// be used with slp related rpc methods and also causes slp metadata to be added
// in some of the existing rpc methods.
bool slp_index = 8;
// When `slp_graphsearch` is true, the node is able to handle calls to slp graph search
bool slp_graphsearch = 9;
}

message GetBlockInfoRequest {
Expand Down Expand Up @@ -569,8 +576,10 @@ message GetSlpTrustedValidationRequest {
message Query {
bytes prev_out_hash = 1;
uint32 prev_out_vout = 2;
repeated bytes graphsearch_valid_hashes = 3;
}
repeated Query queries = 1;
bool include_graphsearch_count = 2;
}

message GetSlpTrustedValidationResponse {
Expand All @@ -585,11 +594,21 @@ message GetSlpTrustedValidationResponse {
bool v1_mint_baton = 7;
}
bytes slp_txn_opreturn = 8;
uint32 graphsearch_txn_count = 9;
}

repeated ValidityResult results = 1;
}

message GetSlpGraphSearchRequest {
bytes hash = 1;
repeated bytes valid_hashes = 2;
}

message GetSlpGraphSearchResponse {
repeated bytes txdata = 1;
}

// NOTIFICATIONS

message BlockNotification {
Expand Down
Loading

0 comments on commit 06690af

Please sign in to comment.