Opening article
TLDR: Going forward, we're aiming for Transformers to be the pivot across frameworks: if a model architecture is supported by transformers, you can expect it to be supported in the rest of the ecosystem. Transformers was created in 2019, shortly following the release of the BERT Transformer model. Transformers now supports 300+ model architectures, with an average of ~3 new architectures added every week. Over time, Transformers has become a central component in the ML ecosystem, becoming one of the most complete toolkits in terms of model diversity; it's integrated in all popular training frameworks such as Axolotl, Unsloth, DeepSpeed, FSDP, PyTorch-Lightning, TRL, Nanotron, etc.
TLDR: Going forward, we're aiming for Transformers to be the pivot across frameworks: if a model architecture is supported by transformers, you can expect it to be supported in the rest of the ecosystem.
Transformers was created in 2019, shortly following the release of the BERT Transformer model. Since then, we've continuously aimed to add state-of-the-art architectures, initially focused on NLP, then growing to Audio and computer vision. Today, transformers is the default library for LLMs and VLMs in the Python ecosystem.
Transformers now supports 300+ model architectures, with an average of ~3 new architectures added every week. We have aimed for these architectures to be released in a timely manner; having day-0 support for the most sought-after architectures (Llamas, Qwens, GLMs, etc.).
We are super proud that the transformers format is being adopted by the community, bringing a lot of interoperability we all benefit from. Train a model with Unsloth, deploy it with SGLang, and export it to llama.cpp to run locally! We aim to keep supporting the community going forward.
We firmly believe this renewed direction will help standardize an ecosystem which is often at risk of fragmentation. We'd love to hear your feedback on the direction the team has decided to take; and of changes we could do to get there. Please come and see us over at the transformers-community support tab on the Hub!
Over time, Transformers has become a central component in the ML ecosystem, becoming one of the most complete toolkits in terms of model diversity; it's integrated in all popular training frameworks such as Axolotl, Unsloth, DeepSpeed, FSDP, PyTorch-Lightning, TRL, Nanotron, etc.
Recently, we've been working hand in hand with the most popular inference engines (vLLM, SGLang, TGI, ...) for them to use transformers as a backend. The value added is significant: as soon as a model is added to transformers , it becomes available in these inference engines, while taking advantage of the strengths each engine provides : inference optimizations, specialized kernels, dynamic batching, etc.
That's all it takes for a new model to enjoy super-fast and production-grade serving with vLLM!
The Transformers Library: standardizing model definitions. Covered by Hugging Face on May 15, 2025.