Surformer: An interpretable pattern-perceptive survival transformer for cancer survival prediction from histopathology whole slide images

Abstract

Background and Objective: High-resolution histopathology whole slide images (WSIs) contain abundant valuable information for cancer prognosis. However, most computational pathology methods for survival prediction have weak interpretability and cannot explain the decision-making processes reasonably. To address this issue, we propose a highly interpretable neural network termed pattern-perceptive survival transformer (Surformer) for cancer survival prediction from WSIs. Methods: Notably, Surformer can quantify specific histological patterns through bag-level labels without any patch/cell-level auxiliary information. Specifically, the proposed ratio-reserved cross-attention module (RRCA) generates global and local features with the learnable prototypes (𝑝𝑔𝑙𝑜𝑏𝑎𝑙, 𝑝𝑙𝑜𝑐𝑎𝑙𝑠) as detectors and quantifies the patches correlative to each 𝑝𝑙𝑜𝑐𝑎𝑙 in the form of ratio factors (rfs). Afterward, multi-head self&crossattention modules proceed with the computation for feature enhancement against noise. Eventually, the designed disentangling loss function guides multiple local features to focus on distinct patterns, thereby assisting 𝑟𝑓 𝑠 from RRCA in achieving more explicit histological feature quantification. Results: Extensive experiments on five TCGA datasets illustrate that Surformer outperforms existing state-ofthe-art methods. In addition, we highlight its interpretation by visualizing rfs distribution across high-risk and low-risk cohorts and retrieving and analyzing critical histological patterns contributing to the survival prediction. Conclusions: Surformer is expected to be exploited as a useful tool for performing histopathology image datadriven analysis and gaining new insights for interpreting the associations between such images and patient survival states.