Ms.fixtest (Chia-Network#432)

* New chiapos API

* Fix flake

* Move to chiapos 0.12.30 to fix segfaults

* Black

* change stripe size to 2000

* chiavdf to 0.12.25

* Revert "chiavdf to 0.12.25"

This reverts commit f7f596c.
chiavdf has a timelord compile error on MacOS.

* Increase default -b

* Plotter GUI test

Co-authored-by: Gene Hoffman <[email protected]>
Co-authored-by: Yostra <[email protected]>

CHANGELOG.md

@@ -13,6 +13,7 @@ for setuptools_scm/PEP 440 reasons.
 - Node peers are now gossiped between nodes with logic to keep connected nodes on disparate internet networks to partially protect from eclipse attacks. This is the second to last step to remove our temporary introducer and migrate to DNS introducers with peer gossip modeled directly off of Bitcoin. This adds a new database of valid peer nodes that will persist across node restarts. This also makes changes to config.yaml's contents.
 - The rate limited wallet library now supports coin aggregation for adding additional funds after the time of creation.
 - Fees are now used in all applicable rate limited wallet calls
+- New parameters for plotting: -r (number of threads) -s (stripe size) -u (number of buckets) in cli and GUI
 
 ### Changed
 - `sh install.sh` was upgraded so that on Ubuntu it will install any needed OS dependencies.
@@ -21,10 +22,16 @@ for setuptools_scm/PEP 440 reasons.
 - The rpc interfaces of all chia services has been refactored, simplified, and had various additional functionality added.
 - Block timestamps are now stored in the wallet database. Both database versions were incremented and databases from previous versions will not work with Beta 14. However, upon re-sync all test chia since Beta 12 should appear in your wallet.
 - All vestigial references to plots.yaml have been removed.
+- The correct amount of memory is used for plotting
+- Multithreading support in chiapos, as well as a new algorithm which is faster and does 70% less IO.
+- Default -b changed to 3072 to improve performance
 
 ### Fixed
 - Temporary space required for each k size was updated with more accurate estimates.
 - Tables in the README.MD were not rendering correctly on Pypi. Thanks again @altendky.
+- Chiapos issue where memory was spiking and increasing
+- Fixed working space estimates so they are exact
+- Log all errors in chiapos
 
 ## [1.0beta13] aka Beta 1.13 - 2020-09-15
 

electron-react/src/pages/Plotter.js

@@ -334,7 +334,7 @@ const CreatePlot = () => {
   const [maxRam, setMaxRam] = React.useState(3072);
   const [numThreads, setNumThreads] = React.useState(2);
   const [numBuckets, setNumBuckets] = React.useState(0);
-  const [stripeSize, setStripeSize] = React.useState(0);
+  const [stripeSize, setStripeSize] = React.useState(65536);
 
   const changePlotSize = event => {
     setPlotSize(event.target.value);
@@ -493,7 +493,7 @@ const CreatePlot = () => {
                   type="number"
                 />
               <FormHelperText id="standard-weight-helper-text">
-              0 is default (recommended)
+              0 automatically chooses bucket count
               </FormHelperText>
               </FormControl>
             </Grid>
@@ -510,9 +510,6 @@ const CreatePlot = () => {
                   label="Colour"
                   type="number"
                 />
-              <FormHelperText id="standard-weight-helper-text">
-              0 is default (recommended)
-              </FormHelperText>
               </FormControl>
             </Grid>
           </Grid>

mypy.ini

@@ -1,5 +1,5 @@
 [mypy]
 ignore_missing_imports = True
 
-[mypy-lib]
+[mypy - lib]
 ignore_errors = True

setup.py

@@ -6,7 +6,7 @@
     "blspy==0.2.3",  # Signature library
     "chiavdf==0.12.24",  # timelord and vdf verification
     "chiabip158==0.16",  # bip158-style wallet filters
-    "chiapos==0.12.29",  # proof of space
+    "chiapos==0.12.30",  # proof of space
     "clvm==0.5.1",
     "clvm_tools==0.1.6",
     "aiohttp==3.6.2",  # HTTP server for full node rpc

src/cmds/plots.py

@@ -21,7 +21,8 @@ def help_message():
     print(f"command can be any of {command_list}")
     print("")
     print(
-        "chia plots create -k [size] -n [number of plots] -b [memory buffer size MiB] -r [number of threads] -u [number of buckets] -s [stripe size]"
+        "chia plots create -k [size] -n [number of plots] -b [memory buffer size MiB] "
+        + "-r [number of threads] -u [number of buckets] -s [stripe size]"
         + " -f [farmer pk] -p [pool pk] -t [tmp dir] -2 [tmp dir 2] -d [final dir]  (creates plots)"
     )
     print("-i [plotid] [-m memo] are available for debugging")
@@ -45,9 +46,7 @@ def make_parser(parser):
     parser.add_argument(
         "-u", "--buckets", help="Number of buckets", type=int, default=0
     )
-    parser.add_argument(
-        "-s", "--stripe_size", help="Stripe size", type=int, default=0
-    )
+    parser.add_argument("-s", "--stripe_size", help="Stripe size", type=int, default=0)
     parser.add_argument(
         "-f",
         "--farmer_public_key",

src/util/block_tools.py

@@ -100,14 +100,17 @@ def __init__(
             args.size = 18
             # Uses many plots for testing, in order to guarantee proofs of space at every height
             args.num = 40
-            args.buffer = 32
+            args.buffer = 100
             args.farmer_public_key = bytes(self.farmer_pk).hex()
             args.pool_public_key = bytes(self.pool_pk).hex()
             args.tmp_dir = temp_dir
             args.tmp2_dir = plot_dir
             args.final_dir = plot_dir
             args.plotid = None
             args.memo = None
+            args.buckets = 0
+            args.stripe_size = 2000
+            args.num_threads = 0
             test_private_keys = [
                 AugSchemeMPL.key_gen(std_hash(bytes([i]))) for i in range(args.num)
             ]

tests/rpc/test_farmer_harvester_rpc.py

@@ -107,8 +107,8 @@ async def have_plots():
                 token_bytes(32),
                 128,
                 0,
+                2000,
                 0,
-                0
             )
 
             res_2 = await client_2.get_plots()