save_and_load_relay_model.md

Save and Load Relay Model

Pay Attention

Bind the constant values(e.g., "weight" or "parameters" of neural network models) first, otherwise the Relay IR you saved won't contain the constant values.

ir_mod["main"] = bind_params_by_name(ir_mod["main"], params)

The Relay text format has better readability than the JSON format, and the consumed time and file size of these 2 methods almost haven't difference, so I prefer the 1st method.

1. Relay Text Format

# Save
with open("xxx.rly", "w") as f:
    f.write(ir_mod.astext())

# Load
with open("xxx.rly") as f:
    ir_mod = tvm.parser.fromtext(f.read())

2. JSON Format

# Save
with open("xxx.json", "w") as f:
    f.write(tvm.ir.save_json(ir_mod))

# Load
with open("xxx.json") as f:
    ir_mod = tvm.ir.load_json(f.read())

Beside the above 2 TVM specific methods, the Python module "pickle" can do this job too. Although the file generated by this method is binary format, but the size is even bigger than those above text format files, and the deserialization time almost haven't difference with those above TVM specific methods, so the only advantage of this method which I think of maybe is that it don't depend on any part of TVM implementation.

# Save
with open("xxx.pickle", "wb") as f:
    f.write(pickle.dumps(ir_mod))

# Load
with open("xxx..pickle", "rb") as f:
    ir_mod = pickle.loads(f.read())