scrnaseq-deep-learning

Deep Learning for scRNA-seq

Hard rules

1. No fabricated citations. Every cited work must resolve to a verifiable
2. No claim without provenance. Every quantitative or factual claim
3. No silent failure. Every script invocation, API call, or tool use must declare its exit status and what to do on non-zero. A skill that silently swallows errors is a violation.

When to use

• You have 5–10 scRNA-seq batches from different conditions or experiments and need a corrected, batch-integrated expression matrix for joint analysis.
• You have one highly labeled dataset (e.g., 10x Genomics PBMC with cell-type labels) and want to predict labels on a new, unlabeled dataset (label transfer).
• You have perturbation data (drug, CRISPR, genotype) and want to predict the expression of the perturbation from a baseline.
• You have 100+ cells and want to use foundation model embeddings (from scGPT or Geneformer) as input to a downstream classifier for a rare cell type.
• You need to predict RNA velocity but the spliced / unspliced counts are too sparse or noisy for standard scVelo.

When NOT to use

• You have a pure visualization goal (UMAP / t-SNE). Use Scanpy or Seurat with standard PCA on HVGs; no need to train a model.
• You only need clustering / annotation. Graph-based clustering (Louvain / Leiden on kNN-UMAP) is sufficient and faster.
• You are doing differential expression (DE) between two conditions. Use DESeq2 or edgeR (they assume counts are Poisson/negative binomial and are calibrated); scVI's DE is not calibrated to statistical thresholds.
• You are inferring trajectory; standard RNA velocity is the established baseline. Deep learning extensions are not yet routine for primary use cases.
• You have <100 cells. Deep models need statistics; use BBKNN or Seurat's method for small experiments.

Prerequisites

# Core scRNA-seq ecosystem
pip install scanpy scvi-tools

# scVI and friends
pip install scvi-tools[extra]  # includes scvi-tools dependencies

# Transformers for Geneformer / scGPT
pip install transformers torch-geometric

# Optional but useful for full model pipeline
pip install torch lightning-pytorch

# GPU check"
python -c "import torch; print(torch.cuda.is_available(), torch.cuda.get_device_name(0) if torch.cuda.is_available() else None)"

Hardware baselines (rough; verify against each repo):

• scVI fine-tuning (10k cells, ~20k genes): 8–16 GB VRAM.
• scGPT embedding extraction (full transformer): 24–40 GB VRAM.
• Geneformer (base model): 32 GB+ VRAM.
• Multi-batch training (50k cells): 32 GB+ VRAM.

Conceptual prerequisites: know that a gene count matrix is a high-dimensional noisy observation (like a document in bag-of-words NLP), and a batch effect is a non-biological covariate that a deep model can learn to regress out. Foundation models pre-train on the entire transcriptome; fine-tuning on your small labeled set works because the model already knows genes co-express at cell-type level.

Core workflow

1. Pick the model

2. Prepare the input Scanpy AnnData object

A deep model runs on a normalized count matrix (not raw counts) and assumes a common gene set across batches.

import scanpy as sc

# Basic QC: quality control for each dataset in the multi-batch case
def standard_qc(adata):
    adata.var['mt'] = adata.var_names.str.startswith('MT-')  # mitochondrial genes
    adata.var['ribo'] = adata.var_names.str.startswith(('RPS', 'RPL'))  # ribosomal
    sc.pp.calculate_qc_metrics(adata, qc_vars=['mt', 'ribo'], percent_top=None, log1p=False, inplace=True)
    adata = adata[adata.obs.pct_counts_mt < 20, :]  # filter high MT
    adata = adata[adata.obs.n_genes_by_counts > 200, :]  # filter low gene cells
    return adata

# Normalization (total-count normalize, log1p) before training
def normalize_adata(adata):
    sc.pp.normalize_total(adata, target_sum=1e4)
    sc.pp.log1p(adata)
    # Choose highly variable genes (HVGs) to improve signal-to-noise
    sc.pp.highly_variable_genes(adata, min_mean=0.0125, max_mean=3, min_disp=0.5)
    adata = adata[:, adata.var.highly_variable]
    return adata

3. Train scVI for batch correction

scVI is the canonical VAE model; it learns a latent space that pools gene expression across biological variation but removes batch effects by conditioning on the batch index.

import scvi
from scanpy import AnnData

# Basic training
model = scvi.model.SCVI(
    adata,
    n_hidden=128,
    n_latent=30,
    gene_likelihood="nb",
    latent_distribution="normal",
    # If you have batch labels:
    # batch_key="batch"  # name of obs column with batch indices
)
model.train(
    max_epochs=300,
    use_gpu=True,
    batch_size=256,
    early_stopping=True,
    check_val_every_n_epoch=10
)

# Encode (get corrected expression)
latent = model.get_latent_representation()
adata.obsm["X_scVI"] = latent

# Save / load
model.save("scVI_model.pth")
model = scvi.model.SCVI.load("scVI_model.pth", adata)

For scANVI (the successor with hierarchical priors), the pattern is similar but uses scvi.model.SCANVI and often includes a classifier on the latent space.

4. scGPT foundation model training

scGPT treats gene expression as a language: each cell is a document, each gene is a word. It's a transformer architecture with masked language modeling.

import scgpt  # pip install scgpt

# Pre-trained or train your own
model = scgpt.SCGPT.load_pretrained("species-Human")  # or Human or Mus_musculus

# Fine-tune on your dataset
model.train(
    adata,
    max_epochs=100,
    batch_size=128,
    # Include batch information for context
    batch_key="batch",
    # Only train the adapter layers or unfreeze certain layers
    freeze=True,
    lr=1e-4
)

# Extract embeddings for downstream classification
adata.obsm["X_scGPT"] = model.get_embeddings(adata)

5. Geneformer fine-tuning

Geneformer is the foundation model pre-trained on 30M cells; it uses a standard transformer architecture.

import torch
from transformers import AutoModelForMaskedLM, AutoTokenizer

# Geneformer pre-trained model
model_id = "bert-base"  # but see geneformer repo for actual ID
model = AutoModelForMaskedLM.from_pretrained(model_id)
# You may need geneformer's own tokenizer
# from geneformer import GeneformerTokenizer
# tok = GeneformerTokenizer.from_pretrained("gennomer/geneformer")

# Tokenization: pad all cells to the same length or use truncation
def tokenize_for_geneformer(adata, max_len=1000):
    # Each gene index becomes a token; pad to max_len
    cell_tokens = adata.X.argmax(axis=1).tolist()  # pseudocode; adjust as needed
    return tok(cell_tokens, padding="max_length", truncation=True, max_length=max_len)

# Training loop: this will vary; check the Geneformer papers for exact architecture
# Typically: fine-tune the CLS token or last hidden state on cell-type classification

6. Label transfer from a reference atlas

scANVI is optimized for this scenario: you train on a reference atlas with cell types, then project new cells onto the same latent space and predict.

# Use ScanVI to train on a reference atlas (e.g., Human Cell Atlas)
model = scvi.model.SCANVI(
    ref_adata,
    labels_key="cell_type",  # reference cell-type labels
    unlabeled_category="unlabeled"  # new datasets will be unlabeled
)
model.train()

# Load a new dataset and project
adata = scvi.data.read("new_dataset.h5ad")
adata.obs["cell_type"] = "unlabeled"
model.setup_anndata(adata)  # setup for inference only
latent = model.get_latent_representation(adata)
# Predict cell types
cell_types = model.predict(adata)

7. Evaluate batch correction

The standard quantitative metric is ASW (Average Silhouette Width) between batches in the latent space:

• Good ASW (~0.8–0.9): batches are well separated in biological, not batch-driven, space.
• Poor ASW (~0.4): batches are still batch-driven; the model failed to learn biological variation.

from sklearn.metrics import silhouette_score
import numpy as np

def evaluate_batch_correction(adata, batch_key, use_rep="X_scVI"):
    """Compute ASW for batch in the specified representation."""
    batches = adata.obs[batch_key].astype("category").cat.codes
    # Use only the HVGs for ASW
    emb = adata.obsm[use_rep]
    emb = emb[:, adata.var.highly_variable]
    return silhouette_score(emb, batches)

Qualitative: UMAP / t-SNE plots should cluster by cell type, not batch.

Code patterns

Train with validation split

from scvi import data
import numpy as np

# Split training / validation
n_cells = adata.n_obs
n_train = int(n_cells * 0.8)
ix = np.arange(n_cells)
np.random.shuffle(ix)
adata_train = adata[ix[:n_train], :].copy()
adata_val = adata[ix[n_train:], :].copy()

# Reformat for scVI (single AnnData)
adata = adata_train.concatenate(adata_val)
adata.obs['is_train'] = ['train'] * len(adata_train) + ['val'] * len(adata_val)

# Train
model = scvi.model.SCVI(adata)
model.train(
    max_epochs=300,
    use_gpu=True,
    batch_size=256,
    check_val_every_n_epoch=10,
    # callbacks can include early stopping
)

Classify on latent space

from sklearn.ensemble import RandomForestClassifier
from sklearn.model_selection import train_test_split

# Split in latent space
X = adata.obsm["X_scVI"]
y = adata.obs["cell_type"].astype('category')
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# Train a classifier on the corrected expression
clf = RandomForestClassifier(n_estimators=100)
clf.fit(X_train, y_train)
score = clf.score(X_test, y_test)
print(f"Accuracy: {score:.3f}")

Foundation model downstream analysis

# Differential gene expression using foundation model embeddings
# Treat gene expression as a language task: are the tokens for this gene different
# between conditions in the attention heads?

# Use the attention patterns to find co-expressed genes
# Example: aggregate attention across all cells for gene pair (i,j)
# High attention -> co-expressed

def coexpressed_genes(model, gene_i, n_return=10):
    """Find genes co-expressed with gene_i using attention weights."""
    # This varies by model; see model-specific doc for attention extraction
    attn_weights = model.get_attention_weights_for_gene(gene_i)
    genes = sorted(zip(attn_weights, model.all_genes), key=lambda x: -x[0])
    return [g[1] for g in genes[:n_return]]

GPU memory saving tricks

# For scVI: enable gradient checkpointing
model = scvi.model.SCVI(adata, use_gpu=True)
model._module.enable_checkpointing()  # if available; depends on version

# For large gene sets: do not train on all genes
# Standard: use top 2000 HVGs

# For Geneformer: use mixed precision training
from torch.cuda.amp import autocast
with autocast():
    outputs = model(input_ids, attention_mask)

Common pitfalls

• Training on batch labels as continuous. scVI expects batch as a categorical index, not a continuous value. One-hot encode or use integers.
• Using raw counts as input. Deep models (except Geneformer / scGPT) assume normalized, log-transformed counts. Do not feed raw UMI counts directly.
• Overfitting on small batch sizes. If you have 5,000 cells, batch size of 256 is fine. If you have 1,000, batch size 128 may overfit; try batch size 64 and monitor training loss.
• Forgetting that batch correction removes biological signal. Aggressive batch correction (e.g., using the batch covariate as a strong prior) can remove real biological differences between experiments. Monitor the corrected space with known markers.
• Geneformer-specific: It uses a fixed vocabulary. Make sure the new dataset uses the same set of genes as the pre-trained model.
• Not saving the model and associated transforms. You will not be able to decode or project new cells later without the same training normalization.
• Ignoring evaluation on a held-out batch. When integrating multiple batches, evaluate on one that was held out of training to ensure the model generalizes.
• Training a VAE and using it directly for DE. VAEs are probabilistic models; the ELBO (evidence lower bound) is not a calibrated p-value. Use standard tools for DE inference.

Validation

• Batch metrics: ASW, BCH (batch clustering entropy), and clustering metrics (NMI with batch labels). If these are not improved over the input space, the model failed.
• Cell-type metrics: If you have cell-type labels, compute accuracy or AUC on a held-out fold in the latent space. Deep models should outperform PCA-based methods.
• Prediction metrics: For label transfer, use held-out labels from the reference (if any) and compute accuracy on the new dataset.
• Downstream tasks: Test model performance (e.g., differential expression, clustering, rare cell type detection) in the corrected space vs. raw space.
• Visualization: Compare UMAP / t-SNE plots: clusters should match known biology in the corrected space and should be more compact / less batch-driven.
• Perturbation prediction: Compare predicted vs observed perturbation effects in a ground-truth validation set.

Open alternatives

• scVI / scANVI: The VAE standard; designed for count data and batch integration.
• Geneformer: Transformer-based foundation model (Theodoris et al., Nature 2023); pre-trained on 30M cells.
• scGPT: Also transformer-based; Liu et al.; includes masking, domain classification, gene importance.
• scFoundation: Another transformer-based model; Zhao et al.; multi-scale architecture.
• totalVI: VAE with protein integration (multi-modal); from the authors of scVI.
• Seurat v4: Uses neural nets (weights learned from scVI) for batch correction in the standard Seurat pipeline.
• Closed alternative: CellVoyager (Veloxity) — commercial package for visualization and clustering.

References

• Lopez et al., 2018, Nature Methods — scVI (the foundational VAE for scRNA-seq).
• Chen et al., 2020 — scANVI (the Bayesian VAE extension).
• Theodoris et al., 2023, Nature — Geneformer (transformer foundation model).
• Liu et al., 2023 — scGPT (transformer with masking).
• Zhao et al., 2023 — scFoundation (multi-scale transformer).
• Yosef et al., 2013, Nature Biotechnology — standard Scanpy reference (not deep learning, but the ecosystem).
• Hafemeister & Satija — Seurat v4 (includes neural net modules).

Related Skills

• bioinformatics-omics/scrnaseq-pipeline — preprocessing, QC, and analysis with traditional methods
• machine-learning-bio/deep-learning-genomics — foundation models for DNA
• machine-learning-bio/protein-language-models — foundation models for protein

Changelog

• 1.0.0 (2026-06-10): Initial adaptation by Pradyumna Jayaram from scVI (Lopez et al. 2018), Geneformer (Theodoris et al. 2023), and scGPT (Liu et al. 2023) papers; added practical fine-tuning workflow, GPU sizing guidelines, batch-correction evaluation rubric, and label transfer patterns.

Cross-references

Other skills in this category:

• alphafold-structure-prediction
• deep-learning-genomics
• protein-language-models