Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* Refactor the training input pipeline. 1. Introduce a binary format (v2). 2. Support for loading v2 format off disk (much faster to parse) 3. Use the v2 format for the shuffle buffer (shrinks it by ~4x) 4. Amend test to be full end-to-end test, including worker interfaces. 5. Add benchmark for the TensorFlow frontend. 6. Change worker->main process communication to be fixed size binary format instead of protobuf. Net result is that the pipeline now saturates at about 45,000 moves/sec into TensorFlow (aka ~88 batches/sec with n=512) Pull request leela-zero#917.
- Loading branch information