Opening article
Arc Institute has unveiled the Virtual Cell Challenge, tasking machine learning engineers with training models to predict the effects of CRISPR gene silencing in partially unseen cell types. To assist participants, Arc released STATE, a baseline system consisting of a State Transition Model and a State Embedding Model. The challenge relies on a curated dataset of approximately 300,000 single-cell RNA sequencing profiles.
Arc Institute recently unveiled the Virtual Cell Challenge . Participants are required to train a model capable of predicting the effect of silencing a gene in a (partially) unseen cell type, a task they term context generalization . For ML engineers with little to no biology background, the jargon and required context can seem quite daunting. To encourage participation, we recapitulate the challenge in a form better suited to engineers from other disciplines.
Goal Train a model to predict the effect on a cell of silencing a gene using CRISPR.
Doing things in the world of atoms is expensive, laborious and error prone. What if we could test thousands of drug candidates without ever touching a petri dish? This is the goal of the virtual cell challenge — a model (most likely a neural network) that can simulate exactly what happens to a cell when we change some parameter. Given that tightening your feedback loop is often the best way to speed up progress, a model capable of doing this accurately would have significant impact.
This inability to measure the cell state before and after introduces many issues, as we are forced to use a population of basal (a.k.a control, unperturbed) cells as a reference point. The control cells and perturbed cells are not entirely homogeneous even prior to the perturbation. This means that we have to now separate out our true signal, the perturbation, from noise induced by the heterogeneity.
Prior to the Virtual Cell Challenge, Arc released STATE , their own attempt to solve the challenge using a pair of transformer based models. This serves as a strong baseline for participants to start with, so we will explore it in detail.
The State Transition Model is a relatively simple transformer with a Llama backbone that operates upon the following:
To produce a single gene embedding, we first obtain the amino acid sequence (e.g ... for TMSB4X) of all the different protein isoforms encoded for by the gene in question. We then feed these sequences to ESM2 , a 15B parameter Protein Language Model from FAIR. ESM produces an embedding per amino acid , and we mean pool them together to obtain a "transcript" (a.k.a protein isoform) embedding.
To train this neural network, we will need data. For the challenge, Arc has curated a dataset of ~300k single-cell RNA sequencing profiles. It may be worthwhile to revisit the Central Dogma before continuing. This essay will build off of this to provide the ~minimum biology knowledge you'll need for the challenge.
The training set consists of a sparse matrix and some associated metadata. More specifically, we have 220k cells, and for each cell we have a transcriptome . This transcriptome is a sparse row vector, where each entry is the raw count of RNA molecules (transcripts) that the corresponding gene (our column) encodes for. Of the 220k cells, ~38k are unperturbed , meaning no gene has been silenced using CRISPR. These control cells are crucial as we will see shortly.
To understand the dataset more concretely, let's select a gene, TMSB4X (the most frequently silenced gene in the dataset) and compare the number of RNA molecules detected for a control cell and a perturbed cell.
Simulating cellular responses using machine learning can drastically reduce the cost and labor of biological experiments, potentially accelerating drug discovery and biological research without touching a petri dish.