Chain Halt

Chain halted at approximately ~4:30 AM PST while processing block 2578098. The network is still in voting rounds currently.

• Human Readable list of transactions in 2578097
• List of all contracts called in 2578097
• Human Readable list of transactions in 2578098
• List of all contracts called in 2578098

UPDATE 9 AM PST 6 April 2022

Root cause has been identified. Two paths are being explored. One path is to restart network with current data folders on top of a new version. The other path is an export and height increment and restart network with same chain-id. The below list is for restarting from export.

Tasks

• Verify that 3 exports from block 97 only differ by the hash generated by the malicous contract (@faddat and team)
• Verify state changes between v2.1.0 and v3.0.0 (nee v2.3.2) and make list that would need to happen to genesis file (@faddat and team)
• Write a junod migrate command that takes the JSON from the export and makes modifications to it according to list of changes needed (@faddat and team, but likely need more hands and eyes on this work)
  • Increment .initial_height in the export to be 99
  • Remove lupercalia upgrade proposal and votes from state
  • additional TBD modifications
• Validate genesis.json (assembled by @gnudrew25 with help of @dimiandre and others) that has these modifications:
  • Increment .initial_height in the export to be 99
  • Change lupercalia upgrade proposal to a text proposal, add notes about halt and changes to upgrade.
  • Reset satoshi-test contract state to {"count":0,"owner":"juno1hxkppd7spnvm5s86z2rfze5pndg9wwee8g9x6v","name":""}
• Test upgrade using ibc test framework to impersonate the top 29 validators (@agouin and @jackzampolin)
  • Make list of their valoper keys
  • Generate the new validator set
  • Find and replace all instances of one validator's key with the new validator's key
  • start the network and ensure that it runs smoothly (network runs smoothly from block 99 on v2.3.1 and v3.0.0-alpha)
  • TBD testing to ensure validatity of ugprade
• Hardfork planning (needs resourcing)
  • Plan timing of network restart
  • Notify validators
  • Write instructions for upgrade