Transformer-based biomarker prediction from colorectal cancer histology: A large-scale multicentric study
Abstract: Deep learning (DL) can accelerate the prediction of prognostic biomarkers from routine pathology slides in colorectal cancer (CRC). However, current approaches rely on convolutional neural networks (CNNs) and have mostly been validated on small patient cohorts. Here, we develop a new transformer-based pipeline for end-to-end biomarker prediction from pathology slides by combining a pre-trained transformer encoder with a transformer network for patch aggregation. Our transformer-based approach substantially improves the performance, generalizability, data efficiency, and interpretability as compared with current state-of-the-art algorithms. After training and evaluating on a large multicenter cohort of over 13,000 patients from 16 colorectal cancer cohorts, we achieve a sensitivity of 0.99 with a negative predictive value of over 0.99 for prediction of microsatellite instability (MSI) on surgical resection specimens. We demonstrate that resection specimen-only training reaches clinical-grade performance on endoscopic biopsy tissue, solving a long-standing diagnostic problem.
![Image Alt Text](/peng-lab/HistoBistro/raw/main/CancerCellCRCTransformer/visualizations/2023-07_graphical-abstract.jpg)
In this folder you find additional resources used for the publication that are not contained in the main repository:
trained_models
: Folder with our models trained on the multicentric dataset for MSI high, BRAF, and KRAS predictionBRAF_CRC_model.pth
KRAS_CRC_model.pth
MSI_high_CRC_model.pth
visualizations
: Folder with resourcesd for the visualizations in the publicationVisualize_results.ipynb
: Jupyter notebook to visualize the results and plot the data overview figureUserStudy.ipynb
: Jupyter notebook to evaluate the user study
config.yaml
: Config file used for training the modelsevaluations_user_study.xlsx
: Table with expert evaluations of high attention tiles from the user studyexperimental_results.xlsx
: Table with all experimental resultstrain_num_samples.py
: Script to train models with different numbers of training samples