SkinGPT-4: An Interactive Dermatology Diagnostic System with Visual Large Language Model

Juexiao Zhou, Xiaonan He, Liyuan Sun, Jiannan Xu, Xiuying Chen, Yuetan Chu, Longxi Zhou, Xingyu Liao, Bin Zhang, Xin Gao

King Abdullah University of Science and Technology, KAUST

Installation

Please refer to the original MiniGPT-4: https://github.com/Vision-CAIR/MiniGPT-4

Download our trained weights

Our primary trained weights for skin disease diagnosis with only step-1 dataset could be downloaded at skinGPT_v1.pth.

Then, set the path to the pretrained checkpoint in the evaluation config file in eval_configs/minigpt4_eval.yaml at Line 11.

To access the latest non-commercial model trained with a much larger in-house dataset, please email: [email protected]

Launching Demo Locally

python demo.py --cfg-path eval_configs/minigpt4_eval.yaml  --gpu-id 0

Illustraion of SkinGPT

Examples of Skin disease diagnosis

Clinical Evaluation

Citation

If you're using SkinGPT-4 in your research or applications, please cite both SkinGPT-4 and MiniGPT-4 using this BibTeX:

@misc{zhou2023skingpt4,
      title={SkinGPT-4: An Interactive Dermatology Diagnostic System with Visual Large Language Model}, 
      author={Juexiao Zhou and Xiaonan He and Liyuan Sun and Jiannan Xu and Xiuying Chen and Yuetan Chu and Longxi Zhou and Xingyu Liao and Bin Zhang and Xin Gao},
      year={2023},
      eprint={2304.10691},
      archivePrefix={arXiv},
      primaryClass={eess.IV}
}

@misc{zhu2022minigpt4,
      title={MiniGPT-4: Enhancing Vision-language Understanding with Advanced Large Language Models}, 
      author={Deyao Zhu and Jun Chen and Xiaoqian Shen and xiang Li and Mohamed Elhoseiny},
      year={2023},
}