Next-Generation Cell Type Annotation

Single-cell RNA sequencing (scRNA-seq) extracts the gene expression profile of each cell. But this data is high-dimensional and complex.

Correctly labeling which type a cell is (T cell, B cell, monocyte...) is critical. Traditional methods don't capture the relationship between genes and cell types well enough.

The thesis's starting point: model this relationship better and raise annotation accuracy.

The core idea: turn gene symbols into embeddings with natural language processing (NLP). Using the tokenizer + embedding parts of BERT, GPT2 and GPT3, each gene symbol is converted to a vector.

These embeddings capture semantic relationships between genes — connections not visible from expression values alone.

Novelty: no other work combines BERT/GPT2/GPT3 tokenizers with autoencoders for cell type annotation.

The whole study was built on two phases — an engineering decision that makes experiments objectively comparable.

Each pipeline takes the same data but with a different input combination. This clearly answers "how much does each component contribute?"

The four steps inside a pipeline:

1. 1Text embeddingGene symbols converted to vectors with an NLP tokenizer
2. 2Outlier detectionZ-Score, Isolation Forest, Local Outlier Factor
3. 3AutoencoderDimensionality reduction + feature extraction with the encoder part
4. 4ML predictionNeural Network, SVM, SGD, Decision Tree, Random Forest

Main dataset: PBMC from the Human Cell Atlas (COVID-19 multi-omic blood atlas). The Human Protein Atlas's 19,751-gene-symbol dataset was used for gene name mapping.

17 cell types, the top 100 expressed genes per cluster, 1700 highly variable genes (HVG) in total.

The highest score was achieved with Pipeline 2 (Text Embedding + Gene Expression) on the Type III (8 genes) construction: F1 = 0.818.

One-hot encoding pipelines scored lowest (~0.23). There is a clear 0.48–0.59 gap versus text embeddings.

• Text-based pipelines clearly outperformed all other approaches.
• GPT3 was the best embedding model; GPT2 also produced better representations than BERT.
• Type III (8 genes) text construction consistently outperformed simpler constructions.
• Using autoencoders improved model performance.
• Outlier detection improved the results.
• Data preprocessing design was more decisive than the chosen ML model.
• Text embeddings effectively captured complex relationships between genes.
• The method produced more accurate results than one-hot encoding approaches.

The results showed that cell types can be predicted with high accuracy from gene symbols + expression levels — meaning it's possible to build a "Model Atlas" like existing data atlas projects.

Future work: different neural network layer designs, using gene "absence" information, and single-cell multi-omic approaches combining scRNA-seq with scATAC-seq.