Metric Mirages in Cell Embeddings

Voices Powered byElevenlabs logo
Connected to paperThis paper is a preprint and has not been certified by peer review

Metric Mirages in Cell Embeddings


Wang, H.; Leskovec, J.; Regev, A.


Although biological studies increasingly rely on embeddings of single cell profiles, the quality of these embeddings can be challenging to assess. Such evaluations are especially important for avoiding mis- leading biological interpretations, assessing the accuracy of integration methods, and establishing the zero-shot capabilities of foundational models. Here, we posit that current evaluation metrics can be highly misleading. We show this by training a three-layer perceptron, Islander, which outperforms all 11 leading embedding methods on a diverse set of cell atlases, but in fact distorts biological structures, limiting its utility for biological discovery. We then present a metric, scGraph, to flag such distortions. Our work should help learn more robust and reliable cell embeddings.

Follow Us on


Add comment