OpenMed has published a complete protein AI pipeline that spans structure prediction, sequence design, and codon optimization. The centerpiece is CodonRoBERTa, a transformer trained on 250,000 E. coli coding sequences. In a head‑to‑head comparison, RoBERTa achieved a perplexity of 4.01 and a Codon Adaptation Index (CAI) correlation of 0.40, outperforming ModernBERT by a factor of six on perplexity and delivering a 16‑fold improvement in CAI correlation when moving from version 1 to version 2. The team trained four production models across 25 species in 55 GPU‑hours, creating a species‑conditioned system that is not available from other open‑source projects. CodonRoBERTa‑base, with 92 million parameters, attains nearly the same perplexity as the large model while using 3.4 times fewer parameters, offering a cost‑effective option for teams without large GPU clusters. The work highlights that masked language modeling alone does not guarantee biological relevance; domain‑specific metrics such as CAI are essential for evaluating codon‑level models. The pipeline integrates established tools—ESMFold for structure prediction and ProteinMPNN for sequence design—while providing a fully open‑source codon optimization component that can accelerate therapeutic mRNA, vaccine, and recombinant protein development.
Training mRNA Language Models Across 25 Species for $165
OpenMed released a protein AI pipeline that predicts 3D structure, designs amino‑acid sequences, and optimizes codons for expression. The core of the work is CodonRoBERTa, a transformer trained on 250,000 E. coli coding sequences. After comparing RoBERTa and ModernBERT, RoBERTa achieved a perplexity of 4.01 and a CAI correlation of 0.40, outperforming ModernBERT by a wide margin. The team scaled the best model to 25 species, training four production models in 55 GPU‑hours and creating a species‑conditioned system not offered by other open‑source projects.
Why it matters
Demonstrating that classic RoBERTa architectures excel on biological codon data shows that domain‑specific training can outweigh newer NLP innovations, guiding future model design for biotech applications.