High-throughput data have become an essential tool for uncovering the mechanisms of cellular diseases (O’Reilly et al., 2014) and play a crucial role in mechanism-based drug discovery (Xia, 2017). These data contain rich biological information on cell lines, pathways, and morphological cell characterizations, providing a discovery method that focuses on any morphological effect of diseased cells (Pegoraro and Misteli, 2017). In particular, images obtained from high-throughput imaging contain critical biological information underlying normal and pathological cellular processes. For example, cells under high-throughput imaging are cultured in multi-well plates with various perturbations, such as small molecules, to systematically investigate their effects on cellular phenotypes. Accordingly, high-throughput cell images capture information about the morphological responses of cells to molecules and their associated biological processes.

However, acquiring information from images is challenging in part due to substantial complexity and heterogeneity in cellular phenotype, which brings obstacles for accurately capturing cellular morphological variations (Caicedo et al., 2016), understanding and treating diseases such as cancer (y Cajal et al., 2020). When quantifying information in high-throughput images, the critical challenge is generating low-dimension vectors that contain the morphological information of cells, known as single-cell representations. Therefore, researchers introduce image-based profiling (Boutros et al., 2015, Usaj et al., 2016, Fetz et al., 2016, Chandrasekaran et al., 2021), which typically involves several steps: image acquisition using high-throughput microscopy techniques, image processing, image feature extraction and feature analysis (Caicedo et al., 2017). Similar to other analysis methods involving thousands of measurements per profile (Mader et al., 2016), it has a wide range of applications (Caicedo et al., 2016, Bougen-Zhukov et al., 2017), including phenotype identification, assessing perturbation effects, uncovering cellular mechanisms, and understanding the functions and functional relationships of genes (Boutros et al., 2015, Usaj et al., 2016).

Generally, there are two kinds of methods, hand-crafted and machine learning, that characterize the cell morphology either implicitly or explicitly (Tatonetti et al., 2012, Li et al., 2015, Libbrecht and Noble, 2015, Wang et al., 2014). Hand-crafted representations can be easily obtained using image analysis tools such as CellProfiler (Carpenter et al., 2006, Stirling et al., 2021). CellProfiler (CP) is widely used to extract well-established morphological features without much human intervention. The obtained representations are so-called interpretable representations as they describe single-cell morphological characteristics from several aspects such as size, orientation, intensity, etc. Every column of the representations describes a specific aspect.

Recently, machine learning has been widely used in the field of cell images (Pachitariu and Stringer, 2022, Krentzel et al., 2023). Representations learned from machine learning models have shown to be a viable alternative (Goldsborough et al., 2017, Lafarge et al., 2019, Fonnegra et al., 2023, Xun et al., 2023) since their underlying ability to capture novel biological information and facilitate advanced phenotype characterization. These representations show excellent image generation performance and outperform the CP representations in several downstream discrimination tasks, such as the mechanism of action (MoA). Though the learned representations via the machine learning method have limited understanding and explainable ability from a biological perspective, the natural generative ability of several machine learning models provides opportunities for researchers to visualize the cell morphological variation with the representation changes, deepening the biological understanding of the representations.

Previous studies have shown the impressive discrimination MoA performance of the CP representations, which is valuable in drug discovery. The biological interpretability of CP representations gives an intuitive linkage with cellular morphology. Thus, the changes in CP representations with a specific drug are promising to show the cell morphological response to the drug, enabling a better comprehension of the counterfactual scenarios and drug effects.

In this paper, we propose the CP2Image, a model composed of a generator and a discriminator, enables learning the mapping from CP morphological representations to the real images. We demonstrate that the CP2Image model generates realistic images by taking only the CP representations as input in the inference process. Measured by Frechet Inception Distance (FID) to evaluate its generating performance, the CP2Image model has demonstrated superiority over the state-of-the-art Variational Autoencoder (VAE) (Kingma and Welling, 2013).

We generalize the architecture of the generator in the CP2Image to ResNet (He et al., 2016), InceptionNet (Szegedy et al., 2015) and Transformer (Lee et al., 2021, Han et al., 2022) and propose CP2Image-Res18, CP2Image-Res50, CP2Image-Res101, CP2Image-Inception and CP2Image-Transformer respectively. All these variants of the CP2Image model generate realistic single-cell images. By comparing the quantitative measurement FID score of the generated images, we figure out the CP2Image-Transformer model has the smallest FID among the variants.

We also investigate how much morphological information has been preserved in the generated images of the CP2Image model by correlating the CP measurements of generated images and original images. We figure out that several CP representations are well-preserved in the generated images, and demonstrate how altering the values of input CP representations, such as nuclei intensity, leads to corresponding nuclei intensity changes in the generated images.

In addition, we study the conditional phenotype of the generated images of CP2Image. We show the interpolation from negative control (DMSO) to a specific mechanism of action (MoA) to report the gradual phenotypic changes of MoAs. We also investigate the CP representations under the negative control and compound with various concentrations, revealing a trajectory from low concentration to high concentration under a compound, and show the concentration-condition generated images of the linear CP representations. We believe this is the first successful attempt at generating high-quality single-cell images solely from CP representations.

A preliminary conference version of this work was presented at MIDL 2023 (Ji et al., 2024). In this version, we extend our work from four perspectives. 1. We extend the discussion of related work to the diffusion model and GAN-based model of image generation. 2. We provide a more detailed description of the CP2Image model. 3. We generalize the architecture of the generator to ResNet, InceptionNet, and VisionTransformer, and propose a CP2Image-Transformer model. The CP2Image-Transformer generates single-cell images with much smaller FID than the original CP2Image. 4. Limitation and discussion of future work has been extended.