Skip to content

Commit

Permalink
Merge v0.2.4 (includes v0.2.3) into Develop (0xPolygonHermez#2376)
Browse files Browse the repository at this point in the history 
* fix null effective_percentage

* fix forkID calculation

* fix script

* generate json-schema + docs for node config file and network_custom

* fix unittest

* Hotfixv0.1.4 to v0.2.0 (0xPolygonHermez#2255)

* Hotfix v0.1.4 to main (0xPolygonHermez#2250)

* fix concurrent web socket writes

* fix eth_syncing

* fix custom trace internal tx call error handling and update prover

* add test to custom tracer depth issue; fix internal call error and gas used

* fix custom tracer for internal tx with error and no more steps after it

* remove debug code

* Make max grpc message size configurable  (0xPolygonHermez#2179)

* make max grpc message size configurable

* fix state tests

* fix tests

* fix tests

* get SequencerNodeURI from SC if empty and not IsTrustedSequencer

* Optimize trace (0xPolygonHermez#2183)

* optimize trace

* fix memory reading

* update docker image

* update prover image

* fix converter

* fix memory

* fix step memory

* fix structlogs

* fix structlogs

* fix structlogs

* fix structlogs

* fix structlogs

* fix structlogs

* fix structlogs

* fix structlogs

* update prover image

* fix struclogs

* fix memory size

* fix memory size

* fix memory size

* refactor memory resize

* refactor memory resize

* move log for the best fitting tx (0xPolygonHermez#2192)

* fix load zkCounters from pool

* remove unnecessary log.info

* add custom tracer support to CREATES opcode without depth increase (0xPolygonHermez#2213)

* logs

* fix getting stateroot from previous batch (GetWIPBatch)

* logs

* Fix GetWipBatch when previous last batch is a forced batch

* fix forcedBatch trusted state

* Revert "fix getting stateroot from previous batch (GetWIPBatch)"

This reverts commit 860f0e7.

* force GHA

* add pool limits (0xPolygonHermez#2189)

* Hotfix/batch l2 data (0xPolygonHermez#2223)

* Fix BatchL2Data

* Force GHA

* remove failed txs from the pool limit check (0xPolygonHermez#2233)

* debug trace by batch number via external rpc requests (0xPolygonHermez#2235)

* fix trace batch remote requests in parallel limitation (0xPolygonHermez#2244)

* Added RPC.TraceBatchUseHTTPS config parameter

* fix executor version

---------

Co-authored-by: tclemos <[email protected]>
Co-authored-by: tclemos <[email protected]>
Co-authored-by: Toni Ramírez <[email protected]>
Co-authored-by: agnusmor <[email protected]>
Co-authored-by: agnusmor <[email protected]>
Co-authored-by: Thiago Coimbra Lemos <[email protected]>

* fix test

* fix test

---------

Co-authored-by: tclemos <[email protected]>
Co-authored-by: tclemos <[email protected]>
Co-authored-by: Toni Ramírez <[email protected]>
Co-authored-by: agnusmor <[email protected]>
Co-authored-by: agnusmor <[email protected]>
Co-authored-by: Thiago Coimbra Lemos <[email protected]>

* Effective GasPrice refactor+fixes (0xPolygonHermez#2247)

* effective GasPrice refactor

* bugs fixes and finalizer tests fixes

* fix typo

* fix calculate effective gasprice percentage

* fix test gas price

* Fix/0xPolygonHermez#2257 effective gas price receipt (0xPolygonHermez#2258)

* effective gas price returned by the rpc in the receipt

* linter

* bugfix: fixing l2blocks timestamp for the fist batch (0xPolygonHermez#2260)

* bugfix: fixing l2blocks timestamp for the fist batch

Signed-off-by: Nikolay Nedkov <[email protected]>

* fix finalizer unit test

---------

Signed-off-by: Nikolay Nedkov <[email protected]>

* add more comments, and removed fields PrivateKeyPath and PrivateKeyPassword from etherman.Config that are not in use

* add info to git action

* add info to git action

* fix github action

* updated comments

* updated comments

* Fix/0xPolygonHermez#2263 gas used (0xPolygonHermez#2264)

* fix fea2scalar and gas used

* suggestion

* fix fea2scalar

* suggestion

* Fix pending tx when duplicate nonce (0xPolygonHermez#2270)

* fix pending tx when duplicate nonce

* set pool.transaction.failed_reason to NULL when updating an existing tx

* add more log details when adding tx to AddrQueue

* fix query to add tx to the pool. Fix lint errors

* change failed_reason for tx discarded due duplicate nonce

* Only return a tx from the pool if tx is in pending status (0xPolygonHermez#2273)

* Return a tx from the pool only if it is

* fix TestGetTransactionByHash

---------

Co-authored-by: agnusmor <[email protected]>

* fix documentation with  config file

* improve: adding check to skip appending effectivePercentage if current forkId is under 5.

Signed-off-by: Nikolay Nedkov <[email protected]>

* Fiex effectiveGasprice unsigned txs with forkId lower than 5 (0xPolygonHermez#2278)

* feat: adding functionality to stop sequencer on specific batch num from config param.

Signed-off-by: Nikolay Nedkov <[email protected]>

* patch: adding print for X-Real-IP in JSON-RPC

Signed-off-by: Nikolay Nedkov <[email protected]>

* Fix checkIfSynced (0xPolygonHermez#2289)

* [Rehashing] Check logs order and fix blockhash and blockNumber in the log conversion (0xPolygonHermez#2280)

* fix and check order

* linter

* flushID synchronizer (0xPolygonHermez#2287)

* FlushID in synchronizer

* linter

* fix logs

* commnets

* executor error refactor (0xPolygonHermez#2299)

* handle invalid rlp ROM error (0xPolygonHermez#2297)

* add maxL2GasPrice (0xPolygonHermez#2294)

* add maxL2GasPrice

* fix

* fix

* add test

* document parameter

* update description

* Error refactor (0xPolygonHermez#2302)

* error refactor

* refactor

* Fix replaced tx as failed when duplicated nonce (0xPolygonHermez#2308)

* Fix UpdateTxStatus for replacedTx

* Fix adding tx with same nonce on AddrQueue

* log reprocess need (0xPolygonHermez#2309)

* log reprocess need

* Update finalizer.go

* Feature/2300 synchronizer detect if executor restart (0xPolygonHermez#2306)

* detect if executor restarts and stop synchonizer

* Update prover images (0xPolygonHermez#2311)

* update prover image

* update prover images

* change executor param

* Update testnet.prover.config.json

* Update test.permissionless.prover.config.json

* Update test.prover.config.json

* Update public.prover.config.json

* prover params

* prover params

* prover params

* update prover images

* add doc, and fix dockers to be able to use snap/restore feature (0xPolygonHermez#2315)

* add doc, and fix dockers to be able to use snap/restore feature

* add doc for snap/restore feature

---------

Co-authored-by: Toni Ramírez <[email protected]>

* Update docker-compose.yml

* Update docker-compose.yml

* do not add tx to the pool in case err != nil

* do not add tx into the pool if a fatal error in the executor happens during pre execution

* fix dbMultiWriteSinglePosition config value

* workarround for the error error closing batch

* workarround for the error error closing batch

* workarround for the error error closing batch

* workaround for the error of closing batch, another case

* `Worker`'s `AddTxTracker` Bug Fix (0xPolygonHermez#2343)

* bugfix: Resolve  Function Bug in Worker Module

Signed-off-by: Nikolay Nedkov <[email protected]>

* improve: improving the wait for pending txs to be for only the txs for the current address.

Signed-off-by: Nikolay Nedkov <[email protected]>

---------

Signed-off-by: Nikolay Nedkov <[email protected]>

* rename config files (0xPolygonHermez#2349)

* fix closing batch + logs (0xPolygonHermez#2348)

* fix closing batch + logs

* fix

* log description

* typo errors

* fix error: failed to store transactions for batch due to duplicate key

* test

* typo

* Update README.md

* Update release.yml

* bugfix: fixing place where we need to increment the wg per address for pending txs

Signed-off-by: Nikolay Nedkov <[email protected]>

* Store batchL2Data when the batch is opened (0xPolygonHermez#2358)

* add GasPriceMarginFactor and MaxGasPrice to eth-tx-manager (0xPolygonHermez#2360)

* add GasPriceMarginFactor and MaxGasPrice to eth-tx-manager

* add logs, fix config

* update config file documentation

---------

Co-authored-by: joanestebanr <[email protected]>

* bugfix: attaching missing TxTracker.From to pending txs to store for forced batches. (0xPolygonHermez#2365)

Signed-off-by: Nikolay Nedkov <[email protected]>

* Update README.md

* improve: adding logs (0xPolygonHermez#2373)

* improve: adding logs

Signed-off-by: Nikolay Nedkov <[email protected]>

* adding more logs

Signed-off-by: Nikolay Nedkov <[email protected]>

* adding more logs okx#2

Signed-off-by: Nikolay Nedkov <[email protected]>

---------

Signed-off-by: Nikolay Nedkov <[email protected]>

* bugfix: fixing finalizer's  handling. (0xPolygonHermez#2375)

Signed-off-by: Nikolay Nedkov <[email protected]>

* Update README.md

---------

Signed-off-by: Nikolay Nedkov <[email protected]>
Co-authored-by: joanestebanr <[email protected]>
Co-authored-by: Alonso Rodriguez <[email protected]>
Co-authored-by: tclemos <[email protected]>
Co-authored-by: tclemos <[email protected]>
Co-authored-by: agnusmor <[email protected]>
Co-authored-by: agnusmor <[email protected]>
Co-authored-by: Thiago Coimbra Lemos <[email protected]>
Co-authored-by: joanestebanr <[email protected]>
Co-authored-by: Nikolay Nedkov <[email protected]>
  • Loading branch information
@ToniRamirezM @joanestebanr
@ARR552 @tclemos @agnusmor @Psykepro
10 people authored Aug 7, 2023
1 parent b27a6cc commit a10a407
Show file tree
Hide file tree
Showing 16 changed files with 349 additions and 37 deletions.
2 changes: 1 addition & 1 deletion Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ $(GENERATE_SCHEMA_DOC): $(VENV_PYTHON)
$(PYTHON) -m pip install json-schema-for-humans

PHONY: config-doc-gen
config-doc-gen: config-doc-node config-doc-custom_network ## Generate config file's json-schema for node and custom_network and documentation
config-doc-gen: config-doc-node config-doc-custom_network ## Generate config file's json-schema for node and custom_network and documentation
#

.PHONY: config-doc-node
Expand Down
8 changes: 8 additions & 0 deletions config/config_test.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -264,6 +264,14 @@ func Test_Defaults(t *testing.T) {
path: "EthTxManager.ForcedGas",
expectedValue: uint64(0),
},
{
path: "EthTxManager.GasPriceMarginFactor",
expectedValue: float64(1),
},
{
path: "EthTxManager.MaxGasPriceLimit",
expectedValue: uint64(0),
},
{
path: "L2GasPriceSuggester.DefaultGasPriceWei",
expectedValue: uint64(2000000000),
Expand Down
2 changes: 2 additions & 0 deletions config/default.go
View file
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,8 @@ MultiGasProvider = false
FrequencyToMonitorTxs = "1s"
WaitTxToBeMined = "2m"
ForcedGas = 0
GasPriceMarginFactor = 1
MaxGasPriceLimit = 0
[RPC]
Host = "0.0.0.0"
Expand Down
2 changes: 1 addition & 1 deletion docs/config-file/node-config-doc.html
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
</pre></div> </div><div id=EthTxManager_FrequencyToMonitorTxs_ex2 class="jumbotron examples"><div class=highlight><pre><span></span><span class=s2>&quot;300ms&quot;</span>
</pre></div> </div> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><a href=#EthTxManager.WaitTxToBeMined onclick="anchorLink('EthTxManager.WaitTxToBeMined')">EthTxManager.WaitTxToBeMined=</a> </div> <span class="badge badge-success default-value">Default: "2m0s"</span><span class="badge badge-dark value-type">Type: string</span><br> <span class=description><p>WaitTxToBeMined time to wait after transaction was sent to the ethereum</p> </span> <br> <div class="badge badge-secondary">Examples:</div> <br><div id=EthTxManager_WaitTxToBeMined_ex1 class="jumbotron examples"><div class=highlight><pre><span></span><span class=s2>&quot;1m&quot;</span>
</pre></div> </div><div id=EthTxManager_WaitTxToBeMined_ex2 class="jumbotron examples"><div class=highlight><pre><span></span><span class=s2>&quot;300ms&quot;</span>
</pre></div> </div> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><a href=#EthTxManager.PrivateKeys onclick="anchorLink('EthTxManager.PrivateKeys')">EthTxManager.PrivateKeys=</a> </div><span class="badge badge-dark value-type">Type: array of object</span><br> <span class=description><p>PrivateKeys defines all the key store files that are going<br> to be read in order to provide the private keys to sign the L1 txs</p> </span> <h4>Each item of this array must be:</h4> <div class=card> <div class="card-body items-definition" id=EthTxManager_PrivateKeys_items> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><!-- None --><a href="#EthTxManager.PrivateKeys.PrivateKeys items.Path" onclick="anchorLink('EthTxManager.PrivateKeys.PrivateKeys items.Path')">EthTxManager.PrivateKeys.PrivateKeys items.Path=</a> </div><span class="badge badge-dark value-type">Type: string</span><br> <span class=description><p>Path is the file path for the key store file</p> </span> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><!-- None --><a href="#EthTxManager.PrivateKeys.PrivateKeys items.Password" onclick="anchorLink('EthTxManager.PrivateKeys.PrivateKeys items.Password')">EthTxManager.PrivateKeys.PrivateKeys items.Password=</a> </div><span class="badge badge-dark value-type">Type: string</span><br> <span class=description><p>Password is the password to decrypt the key store file</p> </span> <hr> </div> </div> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><a href=#EthTxManager.ForcedGas onclick="anchorLink('EthTxManager.ForcedGas')">EthTxManager.ForcedGas=</a> </div> <span class="badge badge-success default-value">Default: 0</span><span class="badge badge-dark value-type">Type: integer</span><br> <span class=description><p>ForcedGas is the amount of gas to be forced in case of gas estimation error</p> </span> <hr> </div> </div> </div> </div> <div class=accordion id=accordionPool> <div class=card> <div class=card-header id=headingPool> <h2 class=mb-0> <button class="btn btn-link property-name-button" type=button data-toggle=collapse data-target=#Pool aria-expanded aria-controls=Pool onclick="setAnchor('#Pool')"><span class=property-name> <div class=breadcrumbs>[<a href=#Pool onclick="anchorLink('Pool')">Pool</a>] </div></span></button> </h2> Pool service configuration </div> <div id=Pool class="collapse property-definition-div" aria-labelledby=headingPool data-parent=#accordionPool> <div class="card-body pl-5"> <div class=breadcrumbs> <!-- None --><!-- None --><a href=#Pool.IntervalToRefreshBlockedAddresses onclick="anchorLink('Pool.IntervalToRefreshBlockedAddresses')">Pool.IntervalToRefreshBlockedAddresses=</a> </div> <span class="badge badge-success default-value">Default: "5m0s"</span><span class="badge badge-dark value-type">Type: string</span><br> <span class=description><p>IntervalToRefreshBlockedAddresses is the time it takes to sync the<br> blocked address list from db to memory</p> </span> <br> <div class="badge badge-secondary">Examples:</div> <br><div id=Pool_IntervalToRefreshBlockedAddresses_ex1 class="jumbotron examples"><div class=highlight><pre><span></span><span class=s2>&quot;1m&quot;</span>
</pre></div> </div> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><a href=#EthTxManager.PrivateKeys onclick="anchorLink('EthTxManager.PrivateKeys')">EthTxManager.PrivateKeys=</a> </div><span class="badge badge-dark value-type">Type: array of object</span><br> <span class=description><p>PrivateKeys defines all the key store files that are going<br> to be read in order to provide the private keys to sign the L1 txs</p> </span> <h4>Each item of this array must be:</h4> <div class=card> <div class="card-body items-definition" id=EthTxManager_PrivateKeys_items> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><!-- None --><a href="#EthTxManager.PrivateKeys.PrivateKeys items.Path" onclick="anchorLink('EthTxManager.PrivateKeys.PrivateKeys items.Path')">EthTxManager.PrivateKeys.PrivateKeys items.Path=</a> </div><span class="badge badge-dark value-type">Type: string</span><br> <span class=description><p>Path is the file path for the key store file</p> </span> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><!-- None --><!-- None --><a href="#EthTxManager.PrivateKeys.PrivateKeys items.Password" onclick="anchorLink('EthTxManager.PrivateKeys.PrivateKeys items.Password')">EthTxManager.PrivateKeys.PrivateKeys items.Password=</a> </div><span class="badge badge-dark value-type">Type: string</span><br> <span class=description><p>Password is the password to decrypt the key store file</p> </span> <hr> </div> </div> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><a href=#EthTxManager.ForcedGas onclick="anchorLink('EthTxManager.ForcedGas')">EthTxManager.ForcedGas=</a> </div> <span class="badge badge-success default-value">Default: 0</span><span class="badge badge-dark value-type">Type: integer</span><br> <span class=description><p>ForcedGas is the amount of gas to be forced in case of gas estimation error</p> </span> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><a href=#EthTxManager.GasPriceMarginFactor onclick="anchorLink('EthTxManager.GasPriceMarginFactor')">EthTxManager.GasPriceMarginFactor=</a> </div> <span class="badge badge-success default-value">Default: 1</span><span class="badge badge-dark value-type">Type: number</span><br> <div class="description collapse" id=collapseDescription_EthTxManager_GasPriceMarginFactor> <p>GasPriceMarginFactor is used to multiply the suggested gas price provided by the network<br> in order to allow a different gas price to be set for all the transactions and making it<br> easier to have the txs prioritized in the pool, default value is 1.</p> <p>ex:<br> suggested gas price: 100<br> GasPriceMarginFactor: 1<br> gas price = 100</p> <p>suggested gas price: 100<br> GasPriceMarginFactor: 1.1<br> gas price = 110</p> </div> <div> <a class="collapse-description-link collapsed" data-toggle=collapse href=#collapseDescription_EthTxManager_GasPriceMarginFactor aria-expanded=false aria-controls=collapseDescriptionEthTxManager_GasPriceMarginFactor></a> </div> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><a href=#EthTxManager.MaxGasPriceLimit onclick="anchorLink('EthTxManager.MaxGasPriceLimit')">EthTxManager.MaxGasPriceLimit=</a> </div> <span class="badge badge-success default-value">Default: 0</span><span class="badge badge-dark value-type">Type: integer</span><br> <div class="description collapse" id=collapseDescription_EthTxManager_MaxGasPriceLimit> <p>MaxGasPriceLimit helps avoiding transactions to be sent over an specified<br> gas price amount, default value is 0, which means no limit.<br> If the gas price provided by the network and adjusted by the GasPriceMarginFactor<br> is greater than this configuration, transaction will have its gas price set to<br> the value configured in this config as the limit.</p> <p>ex:</p> <p>suggested gas price: 100<br> gas price margin factor: 20%<br> max gas price limit: 150<br> tx gas price = 120</p> <p>suggested gas price: 100<br> gas price margin factor: 20%<br> max gas price limit: 110<br> tx gas price = 110</p> </div> <div> <a class="collapse-description-link collapsed" data-toggle=collapse href=#collapseDescription_EthTxManager_MaxGasPriceLimit aria-expanded=false aria-controls=collapseDescriptionEthTxManager_MaxGasPriceLimit></a> </div> <hr> </div> </div> </div> </div> <div class=accordion id=accordionPool> <div class=card> <div class=card-header id=headingPool> <h2 class=mb-0> <button class="btn btn-link property-name-button" type=button data-toggle=collapse data-target=#Pool aria-expanded aria-controls=Pool onclick="setAnchor('#Pool')"><span class=property-name> <div class=breadcrumbs>[<a href=#Pool onclick="anchorLink('Pool')">Pool</a>] </div></span></button> </h2> Pool service configuration </div> <div id=Pool class="collapse property-definition-div" aria-labelledby=headingPool data-parent=#accordionPool> <div class="card-body pl-5"> <div class=breadcrumbs> <!-- None --><!-- None --><a href=#Pool.IntervalToRefreshBlockedAddresses onclick="anchorLink('Pool.IntervalToRefreshBlockedAddresses')">Pool.IntervalToRefreshBlockedAddresses=</a> </div> <span class="badge badge-success default-value">Default: "5m0s"</span><span class="badge badge-dark value-type">Type: string</span><br> <span class=description><p>IntervalToRefreshBlockedAddresses is the time it takes to sync the<br> blocked address list from db to memory</p> </span> <br> <div class="badge badge-secondary">Examples:</div> <br><div id=Pool_IntervalToRefreshBlockedAddresses_ex1 class="jumbotron examples"><div class=highlight><pre><span></span><span class=s2>&quot;1m&quot;</span>
</pre></div> </div><div id=Pool_IntervalToRefreshBlockedAddresses_ex2 class="jumbotron examples"><div class=highlight><pre><span></span><span class=s2>&quot;300ms&quot;</span>
</pre></div> </div> <hr> <div class=breadcrumbs> <!-- None --><!-- None --><a href=#Pool.IntervalToRefreshGasPrices onclick="anchorLink('Pool.IntervalToRefreshGasPrices')">Pool.IntervalToRefreshGasPrices=</a> </div> <span class="badge badge-success default-value">Default: "5s"</span><span class="badge badge-dark value-type">Type: string</span><br> <span class=description><p>IntervalToRefreshGasPrices is the time to wait to refresh the gas prices</p> </span> <br> <div class="badge badge-secondary">Examples:</div> <br><div id=Pool_IntervalToRefreshGasPrices_ex1 class="jumbotron examples"><div class=highlight><pre><span></span><span class=s2>&quot;1m&quot;</span>
</pre></div> </div><div id=Pool_IntervalToRefreshGasPrices_ex2 class="jumbotron examples"><div class=highlight><pre><span></span><span class=s2>&quot;300ms&quot;</span>
Expand Down
Loading

0 comments on commit a10a407

Please sign in to comment.