release small or tiny version for mobile devices

[zhongqiu1245]

Hi, thank you for your amazing job!
Your AsymmetricCroCo3DStereo is based on CroCoNetV2, and your CroCoNetV2 has ViTBase_SmallDecoder version, ViTBase_LargeDecoder version, ...
I'm try to deploy dust3r on mobile devices. But for mobile devices, it will be hard to deploy AsymmetricCroCo3DStereo or dust3r with DUSt3R_ViTLarge_BaseDecoder_xxx, it's too heavy for them.
(And for some unknowing reasons, dust3r can't be converted into onnx and tensorrt, only keep in pytorch.)
Could you release dust3r with ViTBase_SmallDecoder or lighter version?
Thank you in advance!

[gogojjh]

Hi, before encountering this issue, I attempted to convert the model to ONNX but was unsuccessful as well.
Could you please share the specific issues you faced during your conversion process?

[ptoupas]

Hi @zhongqiu1245 @gogojjh, I am also facing an issue with onnx conversion.
Is there any update on that? Have you managed to convert the model in any way?

[ptoupas]

So @zhongqiu1245 and @gogojjh, I think I have found a way to export the AsymmetricCroCo3DStereo into onnx format. For this to happen you need to do the following changes in the dust3r and croco submodule repositories.

1. You need to uncomment this line of code HERE in the curope2d.py file of the croco submodule repo. Be aware that you need to re-install the curope after this change by following the instructions on the README (or as shown below).

   cd croco/models/curope/
   python setup.py build_ext --inplace
   

2. You need to change this line of code HERE in the postprocess.py file, into the following:

   return xyz * (torch.exp(d) - 1)
   

   This is because the originally used torch.expm1 is not a supported operation in onnx.export.
3. You need to comment/remove this line of code HERE in the blocks.py file of the croco submodule repo, and add the following two lines in its place:

   grid_a, grid_b = torch.meshgrid(x, y)
   self.cache_positions[h,w] = torch.stack([grid_a.flatten(), grid_b.flatten()], dim=1)
   

   This is because the originally used torch.cartesian_prod is not a supported operation in onnx.export.

After making these changes you should be able to convert he model into onnx format using torch.onnx.export(...)

[gogojjh]

@ptoupas Great! Let me also try

[Feng1909]

@ptoupas Hello, have you tried to convert it into tensorrt?

[ptoupas]

Hi @Feng1909, I have only converted to onnx. But I assume a conversion from onnx to tensorrt shouldn't be a problem following the official instructions here, right?

[Feng1909]

@ptoupas That's right. Thank you very much!

[smarttowel]

@ptoupas

3. You need to comment/remove this line of code HERE in the blocks.py file of the croco submodule repo, and add the following two lines in its place:
grid_a, grid_b = torch.meshgrid(x, y)

This give me wrong results. I think, x and y should be swapped, like this:

grid_a, grid_b = torch.meshgrid(y, x)