Propagating model type information (input tensors) to rust?

[winding-lines]

@boncheolgu,

My code fails with the following error:

Input tensor  Ok(TensorInfo { name: "normalized_input_image_tensor", element_kind: kTfLiteFloat32, dims: [1, 300, 300, 3] })
Error: ErrorMessage { msg: "Invalid type reference of `kTfLiteUInt8` to the original type `kTfLiteFloat32`" }

This is the relevant code, this is lifted from your example.

pub fn do_it(model_file: &Path, input: &[u8]) -> Result<(), Error> {
    let model = FlatBufferModel::build_from_file(model_file)?;
    let resolver = BuiltinOpResolver::default();

    let builder = InterpreterBuilder::new(&model, &resolver)?;
    let mut interpreter = builder.build()?;

    interpreter.allocate_tensors()?;
    let input_indexes = interpreter.inputs().to_vec();
    eprintln!(
        "Input tensor  {:?}",
        interpreter.tensor_info(input_indexes[0])
    );
    let buffer = interpreter.tensor_data_mut(input_indexes[0])?;
    buffer.copy_from_slice(input);

    println!("=== Pre-invoke Interpreter State ===");
    interpreter.print_state();

    interpreter.invoke()?;

    println!("\n\n=== Post-invoke Interpreter State ===");
    interpreter.print_state();
    Ok(())
}

The error makes a lot of sense and I will play with the API to make fix it. Have you done any thinking on how we could propagate this information to the rust layer?

One option would be to look at the model and generate code for the API. I am not sure if this is something you thought about and discarded or if it's something you have on the roadmap. Would love to collaborate on it, if you are interested.

[boncheolgu]

Currently, tensor's type can be obtained by TensorInfo::element_kind. I didn't understand what you meant by "generate code for the API". Could you explain it more?

[winding-lines]

I am thinking of a macro that would take a model (can be a stand-in for a family of compatible models) and would generate a typed rust interface for it.

I am not sure if this would be useful, what do you think?

[boncheolgu]

I still don't get it what you suggested. In order to generate APIs, models should be given at compile time, but it is not always possible. e.g. FlatBufferModel is created from a file. However, typed interfaces are always welcome. I'll appreciate it if you try that.