(C) PLOS One
This story was originally published by PLOS One and is unaltered.
 . . . . . . . . . .



Cell-DINO: Self-supervised image-based embeddings for cell fluorescent microscopy [1]

['Théo Moutakanni', 'Meta Platforms Inc.', 'Fair', 'Paris', 'Mics', 'Centralesupelec', 'Université Paris-Saclay', 'Gif-Sur-Yvette', 'Camille Couprie', 'Seungeun Yi']

Date: 2026-01 

Accurately quantifying cellular morphology at scale could substantially empower existing single-cell approaches. However, measuring cell morphology remains an active field of research, which has inspired multiple computer vision algorithms over the years. Here, we show that DINOv2, a vision-transformer based, self-supervised algorithm, has a remarkable ability for learning rich representations of cellular morphology without manual annotations or any other type of supervision. We apply DINOv2 to cell phenotyping problems, and compare the performance of resulting models, called Cell-DINO models, on a wide variety of tasks across two publicly available imaging datasets of diverse specifications and biological focus. Compared to supervised and other self-supervised baselines, Cell-DINO models demonstrate improved performance, especially in low annotation regimes. For instance, to classify protein localization using only 1% of annotations on a challenging single-cell dataset, Cell-DINO performs 70% better than a supervised strategy, and 24% better than another self-supervised alternative. The results show that Cell-DINO can support the study of unknown biological variation, including single-cell heterogeneity and relationships between experimental conditions, making it an excellent tool for image-based biological discovery.

Cellular imaging is widely used in biological studies, and often enables quantitative analysis of phenotypic variation through the use of machine learning algorithms. In this work, we introduce Cell-DINO , an adaptation of the DINOv2, self-supervised representation learning strategy, and demonstrate its ability to produce better features for cell phenotyping problems. On the Human Protein Atlas (HPA) dataset, our experiments show the versatility of the approach, outperforming a supervised vision transformer strategy by 20% on average and by 70% when using only 1% of the labels. Compared to other self-supervised strategies, Cell-DINO outperforms MAE and SimCLR on the most challenging tasks, for instance protein localization classification on the HPA single-cell dataset, with 34% and 20% improvement in performance, respectively. Cell-DINO can also be useful to quantify the effects of pharmacological compounds at high-throughput, where the phenotypes of interest cannot be annotated by hand. Cell-DINO improves the ability to predict the mechanism of action of compounds using Cell Painting images, outperforming SimCLR, MAE, and supervised CNN baselines, as well as CellProfiler.

Funding: This work was partially supported by Muscular Dystrophy Association (Development Grant 628114 to WMP) and by the National Institutes of Health (awards K99HG011488 and R00HG011488 to WMP). NM was funded by PD23 OTKA (no. 146881) by the Hungarian Ministry of Innovation. This research was funded by the U.S. National Science Foundation’s Directorate for Biological Sciences under Grant No. 2348683 awarded to JCC. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Data Availability: All the data used in this work has been made publicly available by their creators. The Human Protein Atlas ( https://www.proteinatlas.org/humanproteome/subcellular ) and the Cell Painting Gallery ( https://broadinstitute.github.io/cellpainting-gallery/overview.html ) have been created and are maintained by other institutions. The links above provide more information on how to obtain the original datasets. We downloaded and pre-processed these image datasets, and we provide the transformed version used in our study in BioImage Archive, as follows: Accession: S-BIAD2443; DOI: 10.6019/S-BIAD2443; URL: https://www.ebi.ac.uk/biostudies/bioimages/studies/S-BIAD2443 . Our implementation is an adaptation of DINOv2, which is hosted in the main DINOv2 repository. This implementation also includes code to reproduce the results of the HPA analysis. The code is available in GitHub at https://github.com/facebookresearch/dinov2/blob/main/docs/README_CELL_DINO.md . The code necessary to reproduce the Cell Painting results with the LINCS dataset is publicly available in GitHub at: https://github.com/CaicedoLab/cell-painting-devbench .

Copyright: © 2025 Moutakanni et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

In this work, we propose that Cell-DINO can be used as a general-purpose strategy for transforming images of cells into quantitative phenotypic data (image embeddings) to advance experimental biology with the use of microscopy. We analyzed two publicly available datasets with different technical specifications and biological goals ( Fig 2 ), and obtained results that show how the embeddings produced by Cell-DINO support a wide range of downstream tasks, from sub-cellular to population-level analyses. The datasets and tasks evaluated in our study are representative of phenotyping and image analysis problems that permeate biomedical research practice, and for which extensive prior work exists, including strong classical feature analysis strategies, and specialized deep learning solutions. Out of the scope of this work is the training of foundation models that generalize beyond the reported benchmarks; instead, our focus is to investigate the properties of Cell-DINO and purely self-supervised strategies to extract meaningful information from biological images. We compare Cell-DINO against relevant state-of-the-art solutions, as well as contemporary SSL strategies. Our results indicate that DINOv2 has great potential for powering various specific bioimage analysis applications [ 39 , 41 , 88 ] and to enable image-based biological discovery.

A) Illustration of the Cell-DINO algorithm, which trains two neural networks: a teacher and a student network. The self-supervised objective aims to match the features produced by the teacher (which observes random global views) with those produced by the student (which observes random global and local views). B) Illustration of the processing pipeline using vision transformer networks trained with DINO. The image of a cell is first tokenized as a sequence of local patches, which the transformer processes using self-attention layers. The output are image embeddings used for downstream tasks.

In our previous work, we designed the DINO algorithm [ 35 ] to create foundation models for natural RGB images. Our approach has been shown to discover meaningful factors of variation in image collections, and to achieve state-of-the-art performance in multiple vision problems [ 41 – 44 ]. Innovations introduced with DINOv2 [ 37 ] include the use of new loss functions [ 45 , 46 ] to further bolster performance. Here, we leverage these advances in self-supervision to obtain image-based embeddings that measure as many aspects of cellular morphology as possible, without targeting specific phenotypic properties ahead of time, by applying the DINOv2 training algorithm [ 37 ] to images of cells. We refer to the approach and resulting models as Cell-DINO ( Fig 1 ). Our study shows that Cell-DINO is an effective self-supervised approach capable of uncovering meaningful biology from fluorescent microscopy images without the need for manual annotations or explicit supervision.

Approaches to image analysis in microscopy remained specialized for a long time. Image analysis in biology has historically relied on bespoke workflows to measure one or two features [ 26 ], such as cell counts or cell size, disregarding much of the content present in microscopy images. Over the last decade, image-based profiling appeared with the goal of automating the extraction of large sets of features to quantify morphology more broadly [ 27 , 28 ]. Although widely adopted, this approach still relied on classical features engineered depending on the study. Recently, deep learning has been successful in learning morphological features directly from data, yet these efforts typically relied on extensive manual annotations [ 29 , 30 ] and/or prior knowledge about the biology of interest [ 31 – 33 ], which may limit its use to targeted applications only. By contrast, self-supervised learning (SSL) for vision has recently emerged as a strategy for obtaining rich representations of cellular phenotypes from images [ 34 – 37 ], and to make image-based phenotypes compatible with large-scale perturbation studies where prior knowledge of phenotypic variation may not exist because the goal is to uncover the effect of such perturbations [ 38 – 40 ]. While early results are promising, the comparative performance and utility of various SSL methods has thus far only been explored on single datasets and/or limited sets of tasks. Moreover, despite their potential, SSL methods typically lag behind the task-specific performance of supervised models.

Visual interpretation of cellular phenotypes through microscopy is widely used to drive biomedical discovery across a variety of questions in biology, including the assessment of subcellular protein localization [ 1 , 2 ], mitochondrial phenotypes [ 3 , 4 ], and cell-cycle stages [ 5 , 6 ], as well as chemical [ 7 ] and genetic perturbations [ 8 , 9 ]. Overcoming the biases and limitations of conventional image analysis methods [ 10 – 12 ], machine learning has the potential to yield unprecedentedly rich and comprehensive features of image-based cellular phenotypes, and to establish morphological profiling among the current roster of single-cell omics in its own right [ 13 – 16 ]. Indeed, deep learning now powers robust cell segmentation methods [ 17 – 23 ], as well as cell phenotyping using classification networks [ 24 , 25 ].

Results

We investigate the capabilities of Cell-DINO in two separate image-based cell phenotyping problems: 1) protein localization and 2) compound screening. The goal of the protein localization task is to identify the subcellular compartment in which a protein of interest is observed in images of cells. For this problem, we use the Human Protein Atlas (HPA) dataset [1], which has been used to benchmark machine learning algorithms in two Kaggle competitions [24,25] and also has extensive manual annotations. The goal of the compound screening task on the other hand is to quantify similarities and differences in the response of cells to treatments by detecting relevant morphological changes. For this problem, we used publicly available datasets from the Cell Painting Gallery (CPG) [47] acquired to screen chemical and genetic perturbations in cancer cell lines. The CPG datasets have fewer annotations and are representative of contemporary image-based profiling experiments.

Cell-DINO encodes diverse biological properties of cells We used our Cell-DINO approach to analyze the HPA and CPG datasets (Fig 2), and we observe that it discovers meaningful factors of variation directly from the images in both cases. In particular, we trained individual Vision Transformers (ViT) using the DINOv2 algorithm on HPA field-of-view (FoV) images (containing multiple cells), as well as single-cell (SC) images for both HPA and CPG, after deriving single-cell images via segmentation algorithms optimized for each case (Methods). We find that the resulting Cell-DINO models yield image embeddings that capture the phenotypic diversity of cells in the corresponding dataset. Specifically, we find that Cell-DINO identifies clusters of cell lines in the HPA-FoV dataset (Fig 3A), and cell lines and perturbation type in the CPG datasets as primary factors of variation (Fig 3B). These results illustrate how Cell-DINO learns meaningful biological phenotypes without being explicitly trained to identify or detect them. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 3. UMAP visualizations of image-based embeddings obtained with Cell-DINO Left column (A,C): Image-based embeddings of the HPA-FoV dataset. Right column (B,D): Image-based embeddings of the Cell Painting datasets. Top row (A,B): unprocessed embeddings. Bottom row (C,D): transformed embeddings for downstream analysis. A) Points are fields of view, and the colors reveal the cell line of the sample. B) Points are single cells, and the colors reveal the source study where the cells come from: LINCS [48] and LUAD [9] are A549 cells, while CDRP [49], TAORF [50], and BBBC022 [51] are U2OS cells. Samples from the five studies were used for training. C) Points are embeddings of fields of view after harmonization [52]; colors are protein localization labels. D) Points are well-level aggregated features after batch correction with sphering [73] from the LINCS dataset. Colors are mechanisms of action; names with a are antagonists and names with i are inhibitors). https://doi.org/10.1371/journal.pcbi.1013828.g003 In contrast to supervised models that specialize on specific tasks, the self-supervised learning algorithm DINOv2 may discover phenotypic structure beyond these targeted factors of variation. That could enable the analysis of multiple phenotypic questions based on a single set of rich, task-agnostic features. To explore this, we transformed the Cell-DINO HPA-FoV embeddings using the single-cell data integration algorithm Harmony [52] to remove variation across cell-line clusters. We point out that Harmony receives only cell-line labels, but no information on protein localization labels. Remarkably, visualizing the resulting harmonized embeddings uncovers a coherent substructure of protein localization clusters, supporting the idea that protein localization patterns may be consistent across human cell lines [53] (Fig 3C). This in turn demonstrates Cell-DINO ’s ability to encode biologically meaningful properties of cells in a way that can be flexibly disentangled for downstream analysis. Note that the resulting harmonized HPA-FoV embeddings qualitatively resemble the structure identified by supervised methods that are trained specifically for protein localization while ignoring cell lines [24] (Fig A in S1 Text). We next asked how Cell-DINO ’s embeddings compare to specialized baselines on quantitative tasks.

Cell-DINO outperforms standard supervised ViT Vision transformers are considered today as the leading architecture in computer vision, reaching state-of-the-art performance on a number of tasks, including image classification [54], object detection [55], and semantic segmentation [56], among others. Cell-DINO is trained with self-supervision, requiring no manual annotations to learn a ViT model that computes meaningful image embeddings. Not using labels for training can be considered a disadvantage because the model is not informed about the relevant image patterns for downstream analysis. Contrary to this, we find that Cell-DINO models can outperform standard supervised ViT models by a substantial margin. Our supervised implementation follows the original classification strategy for ViTs [57], leaving out other potential tricks or advanced supervised optimizations that may improve performance. Specifically, we benchmark Cell-DINO ’s performance on several phenotyping tasks in the HPA datasets, which cover a diversity of subcellular protein localization categories observed in multiple cell lines. Note that the results described in this section (Fig 4) were obtained using a routine train/validation split of the HPA datasets, a more canonical use case for most practitioners, whereas the Kaggle competition test images present a specific generalization task [24], which we pursue in separate evaluations in the following section. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 4. Performance evaluation of Cell-DINO on the HPA dataset. All reported results are the F1-scores of the corresponding classification task. There are two classification tasks: protein localization (PL/green shades) and cell-line (CL/blue shades) prediction. Each task is evaluated in two versions of the HPA dataset: field-of-view (dark colors) images and single-cell images (light colors). See legend in the bottom-right. Results are presented in a grid of bar-plots colored according to the task and dataset, and organized by pretraining data (columns) and model initialization (rows). Pre-training data goes from no supervision with any labels (left column), to medium supervision with cell line labels (center column), and strong supervision with protein localization labels (right column). The rows consider random initialization (top row), pre-trained DINOv2 models without self-supervision with cell images (center row), and initialization with DINOv2 pre-trained weights with additional self-supervision with cellular images (bottom row). The average score in the top-right corner of the figure identifies each method with a colored icon according to the ranking from best to worst performance. The results of Cell-DINO fine-tuned are reported in Table C in S1 Text. https://doi.org/10.1371/journal.pcbi.1013828.g004 Prior work on HPA [24,25] trained supervised deep learning methods directly on protein localization labels, which forces the models to ignore other aspects of cellular morphology –such as broader phenotypes that characterize cell lines– to maximize performance. The intuition is that cell-line variation may be a distractor that prevents the correct identification of the phenotype of interest (protein localization), therefore, an optimal model will encode only those features. To test the extent to which Cell-DINO ’s embeddings can compete with supervised models, we approach cell-line classification and protein localization as two separate tasks. To evaluate these tasks, we compare a single Cell-DINO model against task-specialized supervised ViT models. We find that classifiers trained with Cell-DINO embeddings obtain excellent performance in both cell-line and protein-localization classification tasks compared to fully supervised ViT models (Fig 4, first row). This shows that Cell-DINO learns substantially richer embeddings than supervised models, rendering it useful for multiple downstream tasks, whereas supervised models fail to generalize well across tasks. This limitation of supervised models is further compounded by the amount of labeled data required to train such models. Second, Cell-DINO embeddings also exhibit performance superior to DINOv2 models pre-trained using natural images and fine-tuned for cell-line or protein-localization classification respectively (Fig 4, second row). This shows that the self-supervised strategy of Cell-DINO can leverage a cellular imaging dataset to discover more biologically relevant features than what pre-trained, state-of-the-art natural image models can extract. Notably, we find that by initializing the weights of a Cell-DINO model with DINOv2 pre-trained weights (Cell-DINO +), the overall accuracy can be further improved (Fig 4, third row). This result points to an easy way of improving self-supervised results, and we highly recommend its adoption in practice. In the rest of this paper, we compare other baselines and strategies against Cell-DINO instead of Cell-DINO + because the latter has the unfair advantage of being pre-trained with more data (150M+ natural images) to which other methods do not have access.

Cell-DINO is competitive against highly tuned models We next ask whether Cell-DINO models can compete with highly optimized models on specific tasks. In particular, both the HPA-FoV and HPA-SC datasets were originally introduced as part of Kaggle competitions with the objective of automating the classification of protein localization [24]. After hundreds of submissions, the leaderboards are dominated by highly-tuned ensembles of supervised models, leveraging complex losses, custom class weighting, and data curation, rendering this benchmark very challenging for self-supervised approaches. To test the utility of Cell-DINO ’s embeddings in this scenario, we trained a small supervised two-layer classifier on the protein localization task using frozen Cell-DINO embeddings as input. Remarkably, we find that Cell-DINO ranks in the top 8.8% and 11.6% of submissions of FoV and SC competitions, respectively, among hundreds of highly specialized submissions (Fig 5), even though Cell-DINO does not have specific inductive biases to solve this problem. We further compared Cell-DINO embeddings, against embeddings derived from DINOv2, and the 1st and 2nd ranked solutions in the SC and FoV classification challenges, respectively (Bestfitting [24] and DualHead [25]). Our results confirm that Cell-DINO learns features that perform comparably with fully supervised models in solving protein localization classification, exhibiting negligible performance differences (only 0.014 and 0.077 absolute points in the FoV and SC challenges respectively, Fig 5). PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 5. Evaluation of Cell-DINO in the HPA Kaggle competitions. Left: whole image protein classification competition [24]. Right: weakly-supervised, single-cell protein classification competition [25]. Both plots: distribution of performance scores in the second stage evaluation of valid competition submissions. The horizontal axis is the official competition score (F1-score on the test set), and the vertical axis is the frequency of submissions obtaining the corresponding score. The dotted line is the cumulative distribution. Vertical lines represent models evaluated in this work as follows: ImageNet pre-trained DINO (red), a single supervised CNN from the top competitors (green), Cell-DINO (blue). https://doi.org/10.1371/journal.pcbi.1013828.g005

Cell-DINO outperforms alternative self-supervised strategies We compared Cell-DINO to other two main families of SSL methods, including reconstructive autoencoders, and contrastive learning. The Masked Autoencoder (MAE) [36] approach uses an image reconstruction objective and has been previously shown to obtain scalable results on cellular imaging tasks [38]. The SimCLR approach [34] introduces a contrastive loss maximizing the agreement between two different transformations of the same images in the feature space. MAE and SimCLR are considered strong self-supervised approaches on standard computer vision benchmarks; MAE is widely used by the community, and often requires fine-tuning or additional supervision, while SimCLR has been shown to scale well with more data and larger models [58]. Other methods such as MoCo [59] (contrastive), BYOL [60] (self-distillation), and SimSiam [61] (contrastive) fall into one of the main SSL families evaluated in our work, and have been found to perform on par to each other (BYOL, MOCOv2, SimSiam [61]), or to be outperformed by DINO [35] (MoCov2 and BYOL) on natural image benchmarks. As reported in Table 1, we note that the more challenging the task is, the stronger Cell-DINO outperforms these models. In particular, on the most challenging HPA-SC dataset, Cell-DINO achieves a boost of more than 6 points in cell-line classification and 13 points in protein localization. By contrast, without finetuning, MAE achieves poor results on the protein localization task, underperforming Cell-DINO by almost 50%. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Table 1. Comparison to other self-supervised algorithms. Top and middle: F1-scores obtained in the HPA-FoV and HPA-SC datasets, respectively, on the protein localization (PL) classification and cell line (CL) prediction tasks. Bottom: Results using Cell Painting images on the LINCS mechanism-of-action (MoA) prediction task using the Area Under the Precision-Recall Curve (AUPRC) metric [62], and on treatment classification using a subset of the JUMP-CP dataset (accuracy) [63]. https://doi.org/10.1371/journal.pcbi.1013828.t001 We also conducted a quantitative evaluation of the predictive performance of Cell-DINO in the CPG datasets by approaching two different tasks. The first task is mechanism-of-action (MoA) prediction in chemical perturbation experiments, following a transcriptomics-based prediction strategy previously used in a Kaggle competition and adapted for Cell Painting images [62]. The second task is treatment prediction, which serves as an auxiliary task for representation learning [63], and has been adapted in previous work for evaluation in the JUMP Cell Painting dataset [15]. Our results show that Cell-DINO robustly outperforms both SimCLR and MAE SSL algorithms in both tasks (Table 1), and is even able to improve upon the performance of previous state-of-the-art approaches, including classical features and weakly supervised learning [32]. These results confirm that Cell-DINO yields substantially richer representations than alternative approaches, yielding a single set of embeddings that improve performance across multiple tasks.

Cell-DINO outperforms other pre-trained models There are growing efforts to pre-train and share models for cellular image analysis, including cell segmentation models [19,22] and models for feature extraction [32,33,38,83]. Here we compare the performance of Cell-DINO against state-of-the-art pre-trained models for image-based profiling, specifically OpenPhenom [38] and SubCell [33]. These two are representative of recent developments in the field and bring meaningful innovations to bioimage analysis. Specifically, OpenPhenom is a channel-adaptive ViT pre-trained with Cell Painting data using an extension of the MAE algorithm. Also, SubCell is a family of models pre-trained using MAE and multi-task supervised objectives using the HPA dataset. For this evaluation, we use the models released by the authors of these studies to extract features in the benchmark datasets of our study. Table 2 reports the results and the associated details. Note that these models have been pre-trained with different datasets and have different architectures from our models. We adopted their pre-trained models as-is because they have been presented as foundation models that can be used out-of-the-box. In contrast, our models are specific to the datasets used for analysis since our goal is to investigate the capabilities of the Cell-DINO algorithm rather than producing generalizable foundation models. The results indicate that our dataset-specific Cell-DINO models yield better performance in all cases, demonstrating the remarkable ability of our strategy to identify meaningful features from a dataset without supervision. We used OpenPhenom for HPA-FoV and not for HPA-SC because it was trained with multi-cellular image crops. Similarly, we used SubCell for HPA-SC and not for HPA-FoV because the model was trained with single-cells. We used both for the LINCS benchmark because the original studies report adaptations and recommendations for Cell Painting, which we followed in our experiments. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Table 2. Comparison to other state-of-the-art pre-trained models. Top and middle: F1-scores obtained in the HPA-FoV and HPA-SC datasets, respectively, on the protein localization (PL) classification and cell line (CL) prediction tasks. Bottom: Results using Cell Painting images on the LINCS mechanism-of-action prediction task (MoA AUPRC) [62]. https://doi.org/10.1371/journal.pcbi.1013828.t002 In this evaluation, the baseline models may have a disadvantage because of the difference in training datasets; additional results to address these differences in the HPA dataset are reported in the Tables A and B in S1 Text. The results are nevertheless informative to assess the achievements of the Cell-DINO strategy with respect to the state-of-the-art in the field. The results also open opportunities to combine innovations from these models with the robustness of Cell-DINO in future work.

Cell-DINO reduces the dependency on manual annotations We further find that Cell-DINO drastically reduces the need for supervised training. Our competitive results in the HPA-SC Kaggle competition test sets (see Fig 5) were obtained using a 2-layer classifier, containing over seven times fewer supervised parameters than even a single network inside the ensemble of Bestfitting (1.2 vs. parameters) [24,25]. We asked whether Cell-DINO may alleviate the need for manual annotations. Importantly, while early progress in the adoption of deep learning methods for cellular imaging applications has been driven by manual annotation efforts, our ability to generate imaging data vastly outpaces biologists’ ability to manually annotate phenotypes of interest reliably. To realize the full potential of morphological profiling as a high-content omic, we need methods that can leverage imaging data at scale, while requiring little to no supervised signal. As such, we assess the robustness of Cell-DINO and alternative approaches to diminishing numbers of labeled images during training by comparing classification performance as measured by the F1-score (Fig 6). We use a supervised baseline, consisting of a ViT encoder backbone and a linear decoder that are trained end-to-end over reduced sets of labeled images. For SSL methods we train the ViT encoders with all available images – even unlabeled – followed by training a linear layer with the same reduced set of labeled images as the supervised baseline. The results indicate that Cell-DINO affords major advantages when the availability of supervised signal is drastically reduced. SimCLR performs on par or slightly better than Cell-DINO in the FoV classification task. However, on the more challenging HPA-SC dataset, Cell-DINO outperforms both supervised and SSL approaches by a substantial margin with only 1% of the available annotations. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 6. Low shot performance on HPA datasets. Top row: results on the HPA-FoV dataset. Bottom row: results on the HPA-SC dataset. Left column: results of the protein localization classification task. Right column: results of the cell-line prediction task. All plots: the horizontal axis is the percent of labeled images used for training in logarithmic scale. The vertical axis is the F1-score of the corresponding classification task. https://doi.org/10.1371/journal.pcbi.1013828.g006 Note that at the FoV level, a protein localization class is considered valid if at least one cell in the FoV has the localization of interest. It is usually the case that multiple cells exhibit the localization, resulting in more evidence to select the correct class. Protein localization at the single-cell level is generally more challenging because the pattern needs to be recognized in one cell or the correct classification will be missed. In addition, single-cell training images are not fully annotated, but rather have a weak label inherited from the FoV. Thus, a cell may have a weak positive label assigned when it is actually a negative example. This inconsistency makes it harder to learn from single-cell annotations, and this was by design in the original single cell Kaggle competition because of the difficulty and the challenges of annotating single cells manually [25]. In all cases at 100% of labeled sample usage, the supervised model has access to all the annotated data to tune all of its parameters. In contrast, Cell-DINO only trains the classification head with supervision and the rest of the backbone parameters are pre-trained with self-supervision and then frozen (no fine-tuning). This large difference in supervised parameters would give an advantage to the supervised approach. However, the Cell-DINO models meet and sometimes surpass the performance of supervised models, highlighting the benefits of self-supervision. These include data efficiency that maximizes learning without labels first, and prevention of potential overfitting issues that facilitates generalization.

Cell-DINO enables predictions in image-based perturbation experiments Imaging is being extensively used to quantify the effects of pharmacological compounds in high-throughput screens, in which phenotypes of interest cannot be annotated by hand [15]. The CPG datasets are an example of such an application. Here, existing knowledge from chemical biology is used to create predictive models of various aspects of compound bioactivity by looking at the response of cells to perturbations in images [65,66]. We find that Cell-DINO improves the ability of such models to predict one such aspect of particular importance: the mechanism of action (MoA) of compounds based on Cell Painting images of cells treated with them [67]. Specifically, we first trained a Cell-DINO model using a combined Cell Painting dataset, previously curated for representation learning [32]. Next, we computed image embeddings and created well-level representations for the LINCS dataset to predict MoAs [62]. For the prediction of MoA, we leveraged compound annotations from the Connectivity Map project, which were previously used in a Kaggle competition to predict compound MoA based on transcriptional profiles of cells [68]. We adapted the same evaluation protocol for image-based profiling based on the LINCS Cell Painting dataset [62], a library of more than 1,500 FDA approved drugs screened on A549 cells. We then used the same profiling pipeline and classifier models (Methods) to compare three feature representations: engineered features obtained with CellProfiler [69,70], features obtained with the weakly supervised Cell Painting CNN model [32,71], and Cell-DINO features trained on Cell Painting images. The results show that learned embeddings improve MoA classification compared to engineered features, and Cell-DINO ’s embeddings perform better than the CP-CNN model (Fig 7). MoA classification is approached at the well-level; single-cell heterogeneity could potentially improve prediction performance even further. The results are based on the maximum dose; analysis of lower doses are reported in Table D in S1 Text. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 7. Performance evaluation of Cell-DINO on the CPG datasets. A) The evaluation task is mechanism of action (MoA) prediction, where chemical compounds are used to treat cells and Cell Painting images are collected to assess their effect and infer their MoA with computational models. B) Illustration of the steps of the computational workflow to predict MoA from Cell Painting images. The only step that changes in these experiments is feature extraction (highlighted in bold). C) Box plots comparing the distributions of performance for the evaluated feature extraction methods. Horizontal axis: evaluated methods organized by feature type. Vertical axis: performance score according to the area under the precision-recall curve of the multi-class MoA prediction problem [62]. Highlighted numbers in the boxes are the median values of the distribution. https://doi.org/10.1371/journal.pcbi.1013828.g007

Cell-DINO excels at image-based profiling of cellular morphology Image-based profiling of biological conditions aims to uncover the relationships among groups of samples to discover their similarities or differences. Cell-DINO features appear useful for image-based profiling, as indicated by their ability to cluster samples in a biologically meaningful manner (Fig 3). We thus evaluate the ability of Cell-DINO embeddings to correctly match samples that are expected to have the same biological annotations using simple nearest-neighbors search. In the HPA datasets, we approach the protein localization and cell line classification tasks using a nearest neighbor classifier (kNN), in contrast to previous work that uses full CNN models or linear classifiers. This is a significantly more challenging task because without explicitly training a parameterized classifier, the feature space needs to be well aligned between training and test samples. Even under these challenges, Cell-DINO embeddings excel and provide better performance compared to other approaches, especially for the protein localization classification problem at the single-cell level (Table 3). PPT PowerPoint slide

PNG larger image

TIFF original image Download: Table 3. Nearest neighbors search evaluation. F1-scores obtained with the HPA-FoV and HPA-SC datasets on the protein localization (PL) classification and cell-line (CL) prediction tasks. The SSL Type column reports the self-supervised principle used by the method, and the Supervision column indicates the type of labels used by the model during training. https://doi.org/10.1371/journal.pcbi.1013828.t003 We also find that Cell-DINO embeddings contain variations of cellular morphology that capture spatial and signal-intensity heterogeneity in the HPA-FoV images. We investigated this based on annotations available through the HPA web-portal on spatial and intensity variation of protein localization patterns that reflect population wide consensus patterns [72]. We find that Cell-DINO embeddings are able to cluster single-cells according to their expected heterogeneity (Fig 8). To avoid confounding by cell-line dependent variation, we collected single-cell embeddings for U2OS cells in the HPA-SC dataset, and clustered single-cell features for three example proteins annotated as either presenting no heterogeneity (EFHC2), spatial heterogeneity (PSTPIP2) or intensity heterogeneity (NUSAP) in the protein channel at the single-cell level. Consistently, protein localization appears homogeneous for EFHC2, and heterogeneous for PSTPIP2 and NUSAP according to the similarity matrices of single cells in Fig 8. This demonstrates how Cell-DINO embeddings can detect biologically relevant variation in subpopulations of cells. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 8. Image-based profiling of cellular morphology. A) Single-cell heterogeneity analysis on the HPA-SC dataset. Only U2OS cells were selected for this analysis to measure morphological differences for three proteins EFHC2, PSTPIP2, and NUSAP, which have been annotated as exhibiting no heterogeneity, spatial heterogeneity, and intensity heterogeneity, respectively. The matrices display similarity of single cells from images labeled for the corresponding proteins using Cell-DINO embeddings. Color images in the bottom illustrate the composite multi-channel visualization, and green images represent the corresponding protein channel. The arrows point to cells that differ in their protein localization activity, explaining the heterogeneity patterns observed. Larger version available in Fig D in S1 Text) Similarity matrix of treatment-level profiles of aggregated Cell-DINO embeddings for 1,571 compounds in the LINCS dataset. Two compound clusters are highlighted with colored mechanism-of-action annotations on the right-hand side. In all matrices, dark blue indicates high similarity, and light blue or white indicates low similarity. https://doi.org/10.1371/journal.pcbi.1013828.g008 In the CPG datasets, we created perturbation-level profiles for 1,571 compounds in the LINCS Cell Painting dataset by aggregating Cell-DINO embeddings. Visual inspection of the cosine similarity matrix suggests these profiles naturally uncover similarities between perturbations that share similar MoA annotations (Fig 8). The matrix was reordered using hierarchical clustering and displays various sets of compounds grouped together according to their similar morphological effects in cells. Not all the groups found in the matrix are fully annotated with MoA labels and may reveal unknown combinations of MoAs, which gives an opportunity to investigate novel pharmacological relationships. This is in line with quantitative results in Table 1 and Fig 7 that demonstrate Cell-DINO ’s ability to predict MoAs using existing annotations. Together, these results underpin the inherent biological fidelity of Cell-DINO embeddings, and their potential to enable new and complement existing routes for biological discoveries.

Cell-DINO finds meaningful local features A qualitative evaluation of the patch-level features computed by Cell-DINO reveals that they encode meaningful features of cell morphology. We used PCA analysis to reduce the dimensionality of patch-level features for a subset of U2OS cells in the HPA-SC dataset and visualize the top three principal components (PCs) in the RGB color space (Fig 9A). The resulting visualizations show how different cellular compartments and their morphological properties are encoded in separate PCs. For instance, most of the nucleus variation is encoded in PC-1 (red), while the outer cytoplasm is encoded in PC-2 (green), and the inner cytoplasm is encoded in PC-3 (blue). This local-feature organization explains the ability of Cell-DINO to encode biologically relevant properties in the resulting output embeddings. PPT PowerPoint slide

PNG larger image

TIFF original image Download: Fig 9. Patch-level feature maps obtained with Cell-DINO. A) Example images and feature map visualizations obtained using principal component analysis (PCA) on patch-level features after masking the background. The first three principal components (PCs) are mapped to the RGB color space. B) Matching patch-level features across images using cosine similarity. A query token is selected from the input image and the cosine similarity is calculated against all patches of reference images. https://doi.org/10.1371/journal.pcbi.1013828.g009 In addition to general cellular structure, patch-level features also encode fine-grain, subtle morphological patterns. We evaluated the ability of patch-level features to match more specific morphology across images by computing the cosine similarity between one token in a query image and all tokens in a reference image. The results in Fig 9B show that a token from the nucleus in a query cell with protein localization in the nucleoplasm matches strongly and specifically to tokens in the nucleus of a reference cell with the same protein localization. In a reference cell with no protein localization in the nucleoplasm, the features still match to the nucleus tokens preferentially, but with lower intensity. This indicates that the features encode sufficient information to distinguish general cellular structure as well as their specific and meaningful biological variations.

[END]
---
[1] Url: https://journals.plos.org/ploscompbiol/article?id=10.1371/journal.pcbi.1013828

Published and (C) by PLOS One
Content appears here under this condition or license: Creative Commons - Attribution BY 4.0.

via Magical.Fish Gopher News Feeds:
gopher://magical.fish/1/feeds/news/plosone/