English | 简体中文

Live stream: July 21, 2020 at 8 pm BiliBili station，PaddleOCR Introduction

Live address：https://live.bilibili.com/h5/21689802

Introduction

PaddleOCR aims to create rich, leading, and practical OCR tools that help users train better models and apply them into practice.

Recent updates

• 2020.7.15, Add mobile App demo , support both iOS and Android ( based on easyedge and Paddle Lite)
• 2020.7.15, Improve the deployment ability, add the C + + inference , serving deployment. In addtion, the benchmarks of the ultra-lightweight OCR model are provided.
• 2020.7.15, Add several related datasets, data annotation and synthesis tools.
• 2020.7.9 Add a new model to support recognize the character "space".
• 2020.7.9 Add the data augument and learning rate decay strategies during training.
• more

Features

• Ultra-lightweight OCR model, total model size is only 8.6M
  • Single model supports Chinese/English numbers combination recognition, vertical text recognition, long text recognition
  • Detection model DB (4.1M) + recognition model CRNN (4.5M)
• Various text detection algorithms: EAST, DB
• Various text recognition algorithms: Rosetta, CRNN, STAR-Net, RARE
• Support Linux, Windows, MacOS and other systems.

Visualization

More visualization

You can also quickly experience the ultra-lightweight OCR : Online Experience

Mobile DEMO experience (based on EasyEdge and Paddle-Lite, supports iOS and Android systems): Sign in the website to obtain the QR code for installing the App

Also, you can scan the QR code blow to install the App (Android support only)

• OCR Quick Start

Supported Models:

Model Name	Description	Detection Model link	Recognition Model link	Support for space Recognition Model link
db_crnn_mobile	ultra-lightweight OCR model	inference model / pre-trained model	inference model / pre-trained model	inference model / pre-train model
db_crnn_server	General OCR model	inference model / pre-trained model	inference model / pre-trained model	inference model / pre-train model

Tutorials

• Installation
• Quick Start
• Algorithm introduction
  • Text Detection Algorithm
  • Text Recognition Algorithm
  • END-TO-END OCR Algorithm
• Model training/evaluation
  • Text Detection
  • Text Recognition
  • Yml Configuration
  • Tricks
• Deployment
  • Python Inference
  • C++ Inference
  • Serving
  • Mobile
  • Model Quantization and Compression (coming soon)
  • Benchmark
• Datasets
  • General OCR Datasets(Chinese/English)
  • HandWritten_OCR_Datasets(Chinese)
  • Various OCR Datasets(multilingual)
  • Data Annotation Tools
  • Data Synthesis Tools
• FAQ
• Visualization
  • Ultra-lightweight Chinese/English OCR Visualization
  • General Chinese/English OCR Visualization
  • Chinese/English OCR Visualization (Support Space Recognization )
• Community
• References
• License
• Contribution

Text Detection Algorithm

PaddleOCR open source text detection algorithms list:

• EAST(paper)
• DB(paper)
• SAST(paper)(Baidu Self-Research, comming soon)

On the ICDAR2015 dataset, the text detection result is as follows:

Model	Backbone	precision	recall	Hmean	Download link
EAST	ResNet50_vd	88.18%	85.51%	86.82%	Download link
EAST	MobileNetV3	81.67%	79.83%	80.74%	Download link
DB	ResNet50_vd	83.79%	80.65%	82.19%	Download link
DB	MobileNetV3	75.92%	73.18%	74.53%	Download link

For use of LSVT street view dataset with a total of 3w training data，the related configuration and pre-trained models for text detection task are as follows:

Model	Backbone	Configuration file	Pre-trained model
ultra-lightweight OCR model	MobileNetV3	det_mv3_db.yml	Download link
General OCR model	ResNet50_vd	det_r50_vd_db.yml	Download link

• Note: For the training and evaluation of the above DB model, post-processing parameters box_thresh=0.6 and unclip_ratio=1.5 need to be set. If using different datasets and different models for training, these two parameters can be adjusted for better result.

For the training guide and use of PaddleOCR text detection algorithms, please refer to the document Text detection model training/evaluation/prediction

Text Recognition Algorithm

PaddleOCR open-source text recognition algorithms list:

• CRNN(paper)
• Rosetta(paper)
• STAR-Net(paper)
• RARE(paper)
• SRN(paper)(Baidu Self-Research, comming soon)

Refer to DTRB, the training and evaluation result of these above text recognition (using MJSynth and SynthText for training, evaluate on IIIT, SVT, IC03, IC13, IC15, SVTP, CUTE) is as follow:

Model	Backbone	Avg Accuracy	Module combination	Download link
Rosetta	Resnet34_vd	80.24%	rec_r34_vd_none_none_ctc	Download link
Rosetta	MobileNetV3	78.16%	rec_mv3_none_none_ctc	Download link
CRNN	Resnet34_vd	82.20%	rec_r34_vd_none_bilstm_ctc	Download link
CRNN	MobileNetV3	79.37%	rec_mv3_none_bilstm_ctc	Download link
STAR-Net	Resnet34_vd	83.93%	rec_r34_vd_tps_bilstm_ctc	Download link
STAR-Net	MobileNetV3	81.56%	rec_mv3_tps_bilstm_ctc	Download link
RARE	Resnet34_vd	84.90%	rec_r34_vd_tps_bilstm_attn	Download link
RARE	MobileNetV3	83.32%	rec_mv3_tps_bilstm_attn	Download link

We use LSVT dataset and cropout 30w traning data from original photos by using position groundtruth and make some calibration needed. In addition, based on the LSVT corpus, 500w synthetic data is generated to train the model. The related configuration and pre-trained models are as follows:

Model	Backbone	Configuration file	Pre-trained model
ultra-lightweight OCR model	MobileNetV3	rec_chinese_lite_train.yml	Download link
General OCR model	Resnet34_vd	rec_chinese_common_train.yml	Download link

Please refer to the document for training guide and use of PaddleOCR text recognition algorithms Text recognition model training/evaluation/prediction

END-TO-END OCR Algorithm

• End2End-PSL(Baidu Self-Research, comming soon)

Visualization

1.Ultra-lightweight Chinese/English OCR Visualization more

2. General Chinese/English OCR Visualization more

3.Chinese/English OCR Visualization (Space_support) more

FAQ

1. Error when using attention-based recognition model: KeyError: 'predict'

   The inference of recognition model based on attention loss is still being debugged. For Chinese text recognition, it is recommended to choose the recognition model based on CTC loss first. In practice, it is also found that the recognition model based on attention loss is not as effective as the one based on CTC loss.

2. About inference speed

   When there are a lot of texts in the picture, the prediction time will increase. You can use --rec_batch_num to set a smaller prediction batch size. The default value is 30, which can be changed to 10 or other values.

3. Service deployment and mobile deployment

   It is expected that the service deployment based on Serving and the mobile deployment based on Paddle Lite will be released successively in mid-to-late June. Stay tuned for more updates.

4. Release time of self-developed algorithm

   Baidu Self-developed algorithms such as SAST, SRN and end2end PSL will be released in June or July. Please be patient.

more

Community

Scan the QR code below with your wechat and completing the questionnaire, you can access to offical technical exchange group.

License

This project is released under Apache 2.0 license

Contribution

We welcome all the contributions to PaddleOCR and appreciate for your feedback very much.

• Many thanks to Khanh Tran for contributing the English documentation.
• Many thanks to zhangxin for contributing the new visualize function、add .gitgnore and discard set PYTHONPATH manually.
• Many thanks to lyl120117 for contributing the code for printing the network structure.
• Thanks xiangyubo for contributing the handwritten Chinese OCR datasets.
• Thanks authorfu for contributing Android demo and xiadeye contributing iOS demo, respectively.