Predicting the Original Appearance of Damaged Historical Documents

🖼️ Gallery • 📊 HDR28K • 🔥 Model Zoo • 🚧 Installation • 📺 Inference • 📏 Evaluation

🌟 Highlight

• We introduce a Historical Document Repair (HDR) task, which endeavors to predict the original appearance of damaged historical document images.
• We build a large-scale historical document repair dataset, termed HDR28K, which includes 28,552 damaged-repaired image pairs with character-level annotations and multi-style degradation.
• 🔥🔥🔥 We propose a Diffusion-based Historical Document Repair method (DiffHDR), which augments the DDPM framework with semantic and spatial information

📰 News

• 2024.12.17: Release inference code.
• 2024.12.10: 🎉🎉 Our paper is accepted by AAAI2025.

🏗️ TODO List

• Inference Code.
• HDR28K Dataset Release.
• Repair Demo.
• Traning Code. (Maybe release, due to the copyright)

🔥 Model Zoo

Model	chekcpoint	status
DiffHDR	GoogleDrive / BaiduYun:x62f	Released

🚧 Installation

Prerequisites (Recommended)

• Linux
• Python 3.9
• Pytorch 1.13.1
• CUDA 11.7

Environment Setup

Clone this repo:

git clone https://github.com/yeungchenwa/HDR.git

Step 0: Download and install Miniconda from the official website.

Step 1: Create a conda environment and activate it.

conda create -n diffhdr python=3.9 -y
conda activate diffhdr

Step 2: Install related version Pytorch following here.

# Suggested
pip install torch==1.13.1+cu117 torchvision==0.14.1+cu117 torchaudio==0.13.1 --extra-index-url https://download.pytorch.org/whl/cu117

Step 3: Install the required packages.

pip install -r requirements.txt

📺 Inference

Using DiffHDR for damaged historical documents repair (Some examples including damaged images, mask images, and content images are provided in /examples):

sh scripts/inference.sh

• device: CUDA or CPU used for inference,
• image_path: The damaged image path.
• mask_image_path: The masked image path.
• content_image_path: The content image path.
• save_dir: The directory for saving repaired image.
• content_mask_guidance_scale: The guidance scale of content image and masked image.
• degraded_guidance_scale: The guidance scale of damaged image.
• ckpt_path: The unet checkpoint path.
• num_inference_steps: The number of inference steps.

📊 HDR28K

Coming soon ...

📏 Evaluation

Coming soon ...

💙 Acknowledgement

• diffusers

⛔️ Copyright

• This repository can only be used for non-commercial research purposes.
• For commercial use, please contact Prof. Lianwen Jin ([email protected]).
• Copyright 2024, Deep Learning and Vision Computing Lab (DLVC-Lab), South China University of Technology.

📇 Citation

@inproceedings{yang2024fontdiffuser,
  title={Predicting the Original Appearance of Damaged Historical Documents},
  author={Yang, Zhenhua and Peng, Dezhi and Shi, Yongxin and Zhang, Yuyi and Liu, Chongyu and Jin, Lianwen},
  booktitle={Proceedings of the AAAI conference on artificial intelligence},
  year={2025}
}

🌟 Star Rising