シングルセルシーケンシングのための深層学習：細胞の多様性を見る顕微鏡

imageThe history of each living being is written in its genome, which is stored as DNA and present in nearly every cell of the body. No two cells are the same, even if they share the same DNA and cell type, as they still differ in the regulators that control how DNA is expressed by the cell. The human genome consists of 3 billion base pairs spread over 23 chromosomes. Within this vast genetic code, there are approximately 20,000 to 25,000 genes, constituting the protein-coding DNA and accounting for about 1% of the total genome [1]. To explore the functioning of complex systems in our bodies, especially this small coding portion of DNA, a precise sequencing method is necessary, and single-cell sequencing (sc-seq) technology fits this purpose.

In 2013, Nature selected single-cell RNA sequencing as the Method of the Year [2] (Figure 3), highlighting the importance of this method for exploring cellular heterogeneity through the sequencing of DNA and RNA at the individual cell level. Subsequently, numerous tools have emerged for the analysis of single-cell RNA sequencing data. For example, the scRNA-tools database has been compiling software for the analysis of single-cell RNA data since 2016, and by 2021, the database includes over 1000 tools [3]. Among these tools, many involve methods that leverage Deep Learning techniques, which will be the focus of this article – we will explore the pivotal role that Deep Learning, in particular, has played as a key enabler for advancing single-cell sequencing technologies.

Background

Flow of genetic information from DNA to protein in cells

Let’s first go over what exactly cells and sequences are. The cell is the fundamental unit of our bodies and the key to understanding how our bodies function in good health and how molecular dysfunction leads to disease. Our bodies are made of trillions of cells, and nearly every cell contains three genetic information layers: DNA, RNA, and protein. DNA is a long molecule containing the genetic code that makes each person unique. Like a source code, it includes several instructions showing how to make each protein in our bodies. These proteins are the workhorses of the cell that carry out nearly every task necessary for cellular life. For example, the enzymes that catalyze chemical reactions within the cell and DNA polymerases that contribute to DNA replication during cell division, are all proteins. The cell synthesizes proteins in two steps: Transcription and Translation (Figure 1), which are known as gene expression. DNA is first transcribed into RNA, then RNA is translated into protein. We can consider RNA as a messenger between DNA and protein.

imageFigure 1. The central dogma of biologyWhile the cells of our body share the same DNA, they vary in their biological activity. For instance, the distinctions between immune cells and heart cells are determined by the genes that are either activated or deactivated in these cells. Generally, when a gene is activated, it leads to the creation of more RNA copies, resulting in increased protein production. Therefore, as cell types differ based on the quantity and type of RNA/protein molecules synthesized, it becomes intriguing to assess the abundance of these molecules at the single-cell level. This will enable us to investigate the behavior of our DNA  within each cell and attain a high-resolution perspective of the various parts of our bodies.

In general, all single-cell sequencing technologies can be divided into three main steps:

Isolation of single cells from the tissue of interest and extraction of genetic material from each isolated cell

Amplification of genetic material from each isolated cell and library preparation

Sequencing of the library using a next-generation sequencer and data analysis

Navigating through the intricate steps of cellular biology and single-cell sequencing technologies, a pivotal question emerges: How is single-cell sequencing data represented numerically?

Structure of single-cell sequencing data

The structure of single-cell sequencing data takes the form of a matrix (Figure 2), where each row corresponds to a cell that has been sequenced and annotated with a unique barcode. The number of rows equals the total number of cells analyzed in the experiment. On the other hand, each column corresponds to a specific gene. Genes are the functional units of the genome that encode instructions for the synthesis of proteins or other functional molecules. In the case of scRNA seq data, the numerical entries in the matrix represent the expression levels of genes in individual cells. These values indicate the amount of RNA produced from each gene in a particular cell, providing insights into the activity of genes within different cells.

imageFigure 2. Schema of single-cell sequencing dataSingle Cell Sequencing Overview

For more than 150 years, biologists have wanted to identify all the cell types in the human body and classify them into distinct types based on accurate descriptions of their properties. The Human Cell Atlas Project (HCAP), the genetic equivalent of the Human Genome Project [4], is an international collaborative effort to map all the cells in the human body.” We can conceptualize the Human Cell Atlas as a map endeavoring to portray the human body coherently and systematically. Much like Google Maps, which allows us to zoom in for a closer examination of intricate details, the Human Cell Atlas provides insights into spatial information, internal attributes, and even the relationships among elements”, explains Aviv Regev, a computational and systems biologist at the Broad Institute of MIT and Harvard and Executive Vice President and Head of Genentech Research.

This analogy seamlessly aligns with the broader impact of single-cell sequencing, since it allows the analysis of individual cells instead of bulk populations. This technology proves invaluable in addressing intricate biological inquiries related to developmental processes and comprehending heterogeneous cellular or genetic changes under various treatment conditions or disease states. Additionally, it facilitates the identification of novel cell types within a given cellular population. The initiation of the first single-cell RNA sequencing (scRNA-seq) paper in 2009 [5], subsequently designated as the "method of the year" in 2013 [2], marked the genesis of an extensive endeavor to advance both experimental and computational techniques dedicated to unraveling the intricacies of single-cell transcriptomes.

As the technological landscape evolves, the narrative transitions to the advancements in single-cell research, particularly the early focus on single-cell RNA sequencing (scRNA-seq) due to its cost-effectiveness in studying complex cell populations.” In some ways, RNA has always been one of the easiest things to measure,” says Satija [6], a researcher at the New York Genome Center (NYGC).  Yet, the rapid development of single-cell technology has ushered in a new era of possibilities—multimodal single-cell data integration. Recognized as the "Method of the Year 2019" by Nature [7] (Figure 3), this approach allows the measurement of different cellular modalities, including the genome, epigenome, and proteome, within the same cell. The layering of multiple pieces of information provides powerful insights into cellular identity, posing the challenge of effectively modeling and combining datasets generated from multimodal measurements. This integration challenge is met with the introduction of Multi-view learning [8] methods, exploring common variations across modalities. This sophisticated approach, incorporating deep learning techniques, showcases relevant results across various fields, particularly in biology and biomedicine.

Amidst these advancements, a distinct challenge surfaces in the persistent limitation of single-cell RNA sequencing—the loss of spatial information during transcriptome profiling by isolating cells from their original position. Spatially resolved transcriptomics (SRT) emerges as a pivotal solution [9], addressing the challenge by preserving spatial details during the study of complex biological systems. This recognition of spatially resolved transcriptomics as the method of the year 2020 solidifies its place as a critical solution to the challenges inherent in advancing our understanding of complex biological systems.

imageFigure 3. Evolution of single-cell sequencing over timeHaving explored the panorama of single-cell sequencing, let us now delve into the role of deep learning in the context of single-cell sequencing.

Deep Learning on single-cell sequencing

Deep learning is increasingly employed in single-cell analysis due to its capacity to handle the complexity of single-cell sequencing data. In contrast, conventional machine-learning approaches require significant effort to develop a feature engineering strategy, typically designed by domain experts. The deep learning approach, however, autonomously captures relevant characteristics from single-cell sequencing data, addressing the heterogeneity between single-cell sequencing experiments, as well as the associated noise and sparsity in such data. Below are three key reasons for the application of deep learning in single-cell sequencing:

High-Dimensional Data: Single-cell sequencing generates high-dimensional data, with thousands of genes and their expression levels measured for each cell. Deep learning models are adept at capturing complex relationships and patterns within this data, which can be challenging for traditional statistical methods.

Non-Linearity: Single-cell gene expression data is characterized by its inherent nonlinearity between gene expressions and cell-to-cell heterogeneity. Traditional statistical methods encounter difficulties in capturing the non-linear relationships present in single-cell gene expression data. In contrast, deep learning models are flexible and able to learn complex non-linear mappings.

Heterogeneity: Single-cell data is often characterized by diverse cell populations with varying gene expression profiles, presenting a complex landscape. Deep learning models can play a crucial role in identifying, clustering, and characterizing these distinct cell types or subpopulations, thereby facilitating a deeper understanding of cellular heterogeneity within a sample.

As we explore the reasons behind using deep learning in single-cell sequencing data, it leads us to the question: What deep learning architectures are often used in sc-seq data analysis?

Background on Autoencoders

Autoencoders (AEs) stand out among various deep-learning architectures (such as GANs and RNNs) as an especially relied upon method for decoding the complexities of single-cell sequencing data.  Widely employed for dimensionality reduction while preserving the inherent heterogeneity in the single-cell sequencing data. By clustering cells in the reduced-dimensional space generated by autoencoders, researchers can effectively identify and characterize different cell types or subpopulations. This approach enhances our ability to discern and analyze the diverse cellular components within single-cell datasets. In contrast to non-deep learning models, such as principal component analysis (PCA), which are integral components of established scRNA-seq data analysis software like Seurat [10], autoencoders distinguish themselves by uncovering non-linear manifolds. While PCA is constrained to linear transformations, the flexibility of autoencoders to capture complex non-linear mappings makes it an advanced method to find nuanced relationships embedded in single-cell genomics.

To mitigate the overfitting challenge associated with autoencoders, several enhancements to the autoencoder structure have been implemented, specifically tailored to offer advantages in the context of sc-seq data. One notable adaptation often used in the context of sc-seq data is the denoising autoencoder (DAEs), which amplifies the autoencoder's reconstruction capability by introducing noise to the initial network layer. This involves randomly transforming some of its units to zero. The Denoising Autoencoder then reconstructs the input from this intentionally corrupted version, empowering the network to capture more relevant features and preventing it from merely memorizing the input (overfitting). This refinement significantly bolsters the model's resilience against data noise, thereby elevating the quality of the low-dimensional representation of samples (i.e., bottleneck) derived from the sc-seq data.

A third variation of autoencoders frequently employed in sc-seq data analysis is variational autoencoders (VAEs), exemplified by models like scGen [19], scVI [14], scANVI [28], etc. VAEs, as a type of generative model, learn a latent representation distribution of the data. Instead of encoding the data into a vector of p-dimensional latent variables, the data is encoded into two vectors of size p: a vector of means η and a vector of standard deviations σ. VAEs introduce a probabilistic element to the encoding process, facilitating the generation of synthetic single-cell data and offering insights into the diversity within a cell population. This nuanced approach adds another layer of complexity and richness to the exploration of single-cell genomics.

Applications of deep learning in sc-seq data analysis

This section outlines the main applications of deep learning in improving various stages of sc-seq data analysis, highlighting its effectiveness in advancing crucial aspects of the process.

scRNA-seq data imputation and denoising

Single-cell RNA sequencing (scRNA-seq) data encounter inherent challenges, with dropout events being a prominent concern that leads to significant issues—resulting in sparsity within the gene expression matrix, often characterized by a substantial number of zero values. This sparsity significantly shapes downstream bioinformatics analyses. Many of these zero values arise artificially due to deficiencies in sequencing techniques, including problems like inadequate gene expression, low capture rates, sequencing depth, or other technical factors. As a consequence, the observed zero values do not accurately reflect the true underlying expression levels. Hence, not all zeros in scRNA-seq data can be considered mere missing values, deviating from the conventional statistical approach of imputing missing data values. Given the intricate distinction between true and false zero counts, traditional imputation methods with predefined missing values may prove inadequate for scRNA-seq data. For instance, a classical imputation method, like Mean Imputation, might entail substituting these zero values with the average expression level of that gene across all cells. However, this approach runs the risk of oversimplifying the complexities introduced by dropout events in scRNA-seq data, potentially leading to biased interpretations.

ScRNA-seq data imputation methods can be divided into two categories: deep learning–based imputation method and non–deep learning imputation method. The non–deep learning imputation algorithms involve fitting statistical probability models or utilizing the expression matrix for smoothing and diffusion. This simplicity renders it effective for certain types of samples. For example, Wagner et al. [11] utilized the k-nearest neighbors (KNN) method, identifying nearest neighbors between cells and aggregating gene-specific Unique Molecular Identifiers (UMI) counts to impute the gene expression matrix. In contrast, Huang et al. [12] proposed the SVAER algorithm, leveraging gene-to-gene relationships for imputing the gene expression matrix. For larger datasets (comprising tens of thousands or more), high-dimensional, sparse, and complex scRNA-seq data, traditional computational methods face difficulties, often rendering analysis using these methods difficult and infeasible. Consequently, many researchers have turned to designing methods based on deep learning to address these challenges.

Most deep learning algorithms for imputing dropout events are based on autoencoders (AEs). For instance, in 2018, Eraslan et al. [13] introduced the deep count autoencoder (DCA). DCA utilizes a deep autoencoder architecture to address dropout events in single-cell RNA sequencing (scRNA-seq) data. It incorporates a probabilistic layer in the decoder to model the dropout process. This probabilistic layer accommodates the uncertainty associated with dropout events, enabling the model to generate a distribution of possible imputed values. To capture the characteristics of count data in scRNA-seq, DCA models the observed counts as originating from a negative binomial distribution.

Single-cell variational inference (scVI) is another deep learning algorithm introduced by Lopez et al. [14]. ScVI is a probabilistic variational autoencoder (VAE) that combines deep learning and probabilistic modeling to capture the underlying structure of the scRNA-seq data.  ScVI can be used for imputation, denoising, and various other tasks related to the analysis of scRNA-seq data. In contrast to the DCA model, scVI employs Zero-Inflated Negative Binomial (ZINB) distribution in the decoder part to generate a distribution of possible counts for each gene in each cell. The Zero-Inflated Negative Binomial (ZINB) distribution allows modeling the probability of a gene expression being zero (to model dropout events) as well as the distribution of positive values (to model non-zero counts).

Additionally, another study addressed the scRNA-seq data imputation challenge by introducing a recurrent network layer in their model, known as scScope [15]. This novel architecture iteratively performs imputations on zero-valued entries of input scRNA-seq data. The flexibility of scScope's design allows for the iterative improvement of imputed outputs through a chosen number of recurrent steps (T). Noteworthy is the fact that reducing the time recurrence of scScope to one (i.e., T = 1) transforms the model into a traditional autoencoder (AE). As scScope is essentially a modification of traditional AEs, its runtime is comparable to other AE-based models.

It's important to note that the application of deep learning in scRNA-seq data imputation and denoising is particularly advantageous due to its ability to capture non-linear relationships among genes. This contrasts with standard linear approaches, making deep learning more adept at providing informed and accurate imputation strategies in the context of single-cell genomics.

Batch effect removal

Single-cell data is commonly aggregated from diverse experiments that vary in terms of experimental laboratories, protocols, sample compositions, and even technology platforms. These differences result in significant variations or batch effects within the data, posing a challenge in the analysis of biological variations of interest during the process of data integration. To address this issue, it becomes necessary to correct batch effects by removing technical variance when integrating cells from different batches or studies. The first method that appears for batch correction is a linear method based on linear regression such as Limma package [16] that provides the removeBatchEffect function which fits a line