Evaluation model, wer has been 100

[Judy121121]

run python main.py --config ./config/baseline.yaml --device your_device --load-weights path_to_weight.pt --phase test。The model weights have been changed to what you provided。WER is always 100。Please tell me the solution sometime.Have the same problem with your 《Improving Continuous Sign Language Recognition with Adapted Image Models》

[hulianyuyy]

It's most likely that you have set a wrong path to the evaluator (e.g., sclite), and thus the results can't be correctly decoded with WER 100%. Please see if there is any error during the inference process and check the path.

[Judy121121]

Is it OK to use ctcdecode1.0.3 for my environment? The requirement in your article is 0.4, which means that you can run the code and the result is 100% all the time.

[hulianyuyy]

ctddecode 1.0.3 on my machine is OK. Have you used sclite during inference?

[Judy121121]

I've never used sclite，The code still does not report errors, wer is always 100%。I tried ctccode0.4, could you send me the required environment for ctccode0.4? thank you.

[hulianyuyy]

I use python 3.8, torch==1.13, torchvision==0.14.0, matplotlib==3.4.3, numpy==1.20.3, opencv_python==4.5.5.64, pandas==1.3.4, Pillow==9.4.0, PyYAML==6.0, scipy==1.7.1, six==1.16.0, tqdm==4.62.3. If you still have problems, you could paste the screen shot here.

[Judy121121]

the wer always 100%。

[hulianyuyy]

The code has implied that ./software/sclite not found. You need to install sclite to perform inference. You can install it here, or use python to perform inference for convenience (by setting 'evaluate_tool' as 'python' in line 16 of ./configs/baseline.yaml).

[Judy121121]

thank you.