Schmauch, B.; Herpin, L.; Olivier, A.; Duboudin, T.; Dubois, R.; Gillet, L.; Schiratti, J.-B.; Di Proietto, V.; Le Corre, D.; Bourgoin, A.; Taïeb, J.; Emile, J.-F.; Fridman, W. H.; Pronier, E.; Laurent-Puig, P.; Durand, E. Y.

Spatial Transcriptomics (spTx) offers unprecedented insights into the spatial arrangement of the tumor microenvironment, tumor initiation/progression and identification of new therapeutic target candidates. However, spTx remains complex and unlikely to be routinely used in the near future. Hematoxylin and eosin (H&E) stained histological slides, on the other hand, are routinely generated for a large fraction of cancer patients. Here, we present a novel deep learning-based approach for multiscale integration of spTx with tumor morphology (MISO). We trained MISO to predict spTx from H&E on a new unpublished dataset of 72 10X Genomics Visium samples, and derived a novel estimate of the upper bound on the achievable performance. We demonstrate that MISO enables near single-cell-resolution, spatially-resolved gene expression prediction from H&E. In addition, MISO provides an effective patient representation framework that enables downstream predictive tasks such as molecular phenotyping or MSI prediction.