We aim to establish a unified benchmark for training and evaluating models in scene text detection and recognition. Building on this benchmark, we introduce a general OCR system with accuracy and efficiency, OpenOCR. This repository also serves as the official codebase of the OCR team from the FVL Laboratory, Fudan University.

We sincerely welcome the researcher to recommend OCR or relevant algorithms and point out any potential factual errors or bugs. Upon receiving the suggestions, we will promptly evaluate and critically reproduce them. We look forward to collaborating with you to advance the development of OpenOCR and continuously contribute to the OCR community!

• 🔥OpenOCR: A general OCR system with accuracy and efficiency
  • ⚡[Quick Start] [Model] [ModelScope Demo] [Hugging Face Demo] [Local Demo] [PaddleOCR Implementation]
  • Introduction
    • A practical OCR system building on SVTRv2.
    • Outperforms PP-OCRv4 baseline by 4.5% on the OCR competition leaderboard in terms of accuracy, while preserving quite similar inference speed.
    • Supports Chinese and English text detection and recognition.
    • Provides server model and mobile model.
    • Fine-tunes OpenOCR on a custom dataset: Fine-tuning Det, Fine-tuning Rec.
    • ONNX model export for wider compatibility.
• 🔥SVTRv2: CTC Beats Encoder-Decoder Models in Scene Text Recognition
  • [Paper] [Doc] [Model] [Datasets] [Config, Training and Inference] [Benchmark]
  • Introduction
    • A unified training and evaluation benchmark (on top of Union14M) for Scene Text Recognition
    • Supports 24 Scene Text Recognition methods trained from scratch on the large-scale real dataset Union14M-L-Filter, and will continue to add the latest methods.
    • Improves accuracy by 20-30% compared to models trained based on synthetic datasets.
    • Towards Arbitrary-Shaped Text Recognition and Language modeling with a Single Visual Model.
    • Surpasses Attention-based Encoder-Decoder Methods across challenging scenarios in terms of accuracy and speed
  • Get Started with training a SOTA Scene Text Recognition model from scratch.

• SMTR&FocalSVTR (Yongkun Du, Zhineng Chen*, Caiyan Jia, Xieping Gao, Yu-Gang Jiang. Out of Length Text Recognition with Sub-String Matching, AAAI 2025. Doc, Paper)
• DPTR (Shuai Zhao, Yongkun Du, Zhineng Chen*, Yu-Gang Jiang. Decoder Pre-Training with only Text for Scene Text Recognition, ACM MM 2024. Paper)
• IGTR (Yongkun Du, Zhineng Chen*, Yuchen Su, Caiyan Jia, Yu-Gang Jiang. Instruction-Guided Scene Text Recognition, TPAMI 2025. Doc, Paper)
• SVTRv2 (Yongkun Du, Zhineng Chen*, Hongtao Xie, Caiyan Jia, Yu-Gang Jiang. SVTRv2: CTC Beats Encoder-Decoder Models in Scene Text Recognition, 2024. Doc, Paper)
• CDistNet (Tianlun Zheng, Zhineng Chen*, Shancheng Fang, Hongtao Xie, Yu-Gang Jiang. CDistNet: Perceiving Multi-Domain Character Distance for Robust Text Recognition, IJCV 2024. Paper)
• MRN (Tianlun Zheng, Zhineng Chen*, Bingchen Huang, Wei Zhang, Yu-Gang Jiang. MRN: Multiplexed Routing Network for Incremental Multilingual Text Recognition, ICCV 2023. Paper, Code)
• TPS++ (Tianlun Zheng, Zhineng Chen*, Jinfeng Bai, Hongtao Xie, Yu-Gang Jiang. TPS++: Attention-Enhanced Thin-Plate Spline for Scene Text Recognition, IJCAI 2023. Paper, Code)
• CPPD (Yongkun Du, Zhineng Chen*, Caiyan Jia, Xiaoting Yin, Chenxia Li, Yuning Du, Yu-Gang Jiang. Context Perception Parallel Decoder for Scene Text Recognition, TPAMI (accepted). PaddleOCR Doc, Paper)
• SVTR (Yongkun Du, Zhineng Chen*, Caiyan Jia, Xiaoting Yin, Tianlun Zheng, Chenxia Li, Yuning Du, Yu-Gang Jiang. SVTR: Scene Text Recognition with a Single Visual Model, IJCAI 2022 (Long). PaddleOCR Doc, Paper)
• NRTR (Fenfen Sheng, Zhineng Chen, Bo Xu. NRTR: A No-Recurrence Sequence-to-Sequence Model For Scene Text Recognition, ICDAR 2019. Paper)

• 2025.03.24: 🔥 Releasing the feature of fine-tuning OpenOCR on a custom dataset: Fine-tuning Det, Fine-tuning Rec

• 2025.03.23: 🔥 Releasing the feature of ONNX model export for wider compatibility.

• 2025.02.22: Our paper CPPD is accepted by TPAMI. Accessible in Doc and PaddleOCR Doc.

• 2024.12.31: Our paper IGTR is accepted by TPAMI. Accessible in Doc.

• 2024.12.16: Our paper SMTR is accepted by AAAI 2025. Accessible in Doc.

• 2024.12.03: The pre-training code for DPTR is merged.

• 🔥 2024.11.23 release notes:

  • OpenOCR: A general OCR system with accuracy and efficiency
    • ⚡[Quick Start] [Model] [ModelScope Demo] [Hugging Face Demo] [Local Demo] [PaddleOCR Implementation]
    • Introduction
  • SVTRv2: CTC Beats Encoder-Decoder Models in Scene Text Recognition
    • [Paper] [Doc] [Model] [Datasets] [Config, Training and Inference] [Benchmark]
    • Introduction
    • Get Started with training a SOTA Scene Text Recognition model from scratch.

Note: OpenOCR supports inference using both the ONNX and Torch frameworks, with the dependency environments for the two frameworks being isolated. When using ONNX for inference, there is no need to install Torch, and vice versa.

pip install openocr-python
pip install onnxruntime

from openocr import OpenOCR
onnx_engine = OpenOCR(backend='onnx', device='cpu')
img_path = '/path/img_path or /path/img_file'
result, elapse = onnx_engine(img_path)

• PyTorch version >= 1.13.0
• Python version >= 3.7

conda create -n openocr python==3.8
conda activate openocr
# install gpu version torch
conda install pytorch==2.2.0 torchvision==0.17.0 torchaudio==2.2.0 pytorch-cuda=11.8 -c pytorch -c nvidia
# or cpu version
conda install pytorch torchvision torchaudio cpuonly -c pytorch

After installing dependencies, the following two installation methods are available. Either one can be chosen.

Install OpenOCR:

pip install openocr-python

Usage:

from openocr import OpenOCR
engine = OpenOCR()
img_path = '/path/img_path or /path/img_file'
result, elapse = engine(img_path)

# Server mode
# engine = OpenOCR(mode='server')

git clone https://github.com/Topdu/OpenOCR.git
cd OpenOCR
pip install -r requirements.txt
wget https://github.com/Topdu/OpenOCR/releases/download/develop0.0.1/openocr_det_repvit_ch.pth
wget https://github.com/Topdu/OpenOCR/releases/download/develop0.0.1/openocr_repsvtr_ch.pth
# Rec Server model
# wget https://github.com/Topdu/OpenOCR/releases/download/develop0.0.1/openocr_svtrv2_ch.pth

Usage:

# OpenOCR system: Det + Rec model
python tools/infer_e2e.py --img_path=/path/img_fold or /path/img_file
# Det model
python tools/infer_det.py --c ./configs/det/dbnet/repvit_db.yml --o Global.infer_img=/path/img_fold or /path/img_file
# Rec model
python tools/infer_rec.py --c ./configs/rec/svtrv2/repsvtr_ch.yml --o Global.infer_img=/path/img_fold or /path/img_file

pip install onnx
python tools/toonnx.py --c configs/rec/svtrv2/repsvtr_ch.yml --o Global.device=cpu
python tools/toonnx.py --c configs/det/dbnet/repvit_db.yml --o Global.device=cpu

pip install onnxruntime
# OpenOCR system: Det + Rec model
python tools/infer_e2e.py --img_path=/path/img_fold or /path/img_file --backend=onnx --device=cpu
# Det model
python tools/infer_det.py --c ./configs/det/dbnet/repvit_db.yml --o Global.backend=onnx Global.device=cpu Global.infer_img=/path/img_fold or /path/img_file
# Rec model
python tools/infer_rec.py --c ./configs/rec/svtrv2/repsvtr_ch.yml --o Global.backend=onnx Global.device=cpu Global.infer_img=/path/img_fold or /path/img_file

pip install gradio==4.20.0
wget https://github.com/Topdu/OpenOCR/releases/download/develop0.0.1/OCR_e2e_img.tar
tar xf OCR_e2e_img.tar
# start demo
python demo_gradio.py

Yiming Lei (pretto0), Xingsong Ye (YesianRohn), and Shuai Zhao (fd-zs) from the FVL Laboratory, Fudan University, with guidance from Dr. Zhineng Chen (Homepage), completed the majority work of the algorithm reproduction. Grateful for their outstanding contributions.

TODO

TODO

If you find our method useful for your reserach, please cite:

@article{Du2024SVTRv2,
      title={SVTRv2: CTC Beats Encoder-Decoder Models in Scene Text Recognition},
      author={Yongkun Du and Zhineng Chen and Hongtao Xie and Caiyan Jia and Yu-Gang Jiang},
      journal={CoRR},
      volume={abs/2411.15858},
      eprinttype={arXiv},
      year={2024},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2411.15858}
}

This codebase is built based on the PaddleOCR, PytorchOCR, and MMOCR. Thanks for their awesome work!