Opening article
Transformers has released version 5.0.0rc-0, marking a five‑year gap since the last release candidate. The library now sees over 3 million daily installs, up from 20 000 in v4, and has surpassed 1.2 billion total installs. The ecosystem has grown from 40 to over 400 model architectures, with more than 750 000 model checkpoints on the Hub, compared to roughly 1 000 at v4. The new release focuses on simplicity, modular design, and a shift to PyTorch as the sole backend, while sunsetting Flax/TensorFlow support. It introduces new inference APIs, specialized kernels, and tighter integration with tools such as vLLM, SGLang, and TensorRT LLM.
Transformers' version v4.0.0rc-1, the initial release candidate for version 4, was released on November 19th, 2020. Five years later, we now release v5.0.0rc-0.
Today, as we launch v5, Transformers is installed more than 3 million times each day via pip - up from 20,000/day in v4 🤯. Altogether, it has now surpassed 1.2 billion installs !
The ecosystem has expanded from 40 model architectures in v4 to over 400 today , and the community has contributed more than 750,000 model checkpoints on the Hub compatible with Transformers, up from roughly 1,000 at the time of v4.
Transformers is the backbone of hundreds of thousands of projects, Unsloth included. We build on Transformers to help people fine-tune and train models efficiently, whether that’s BERT, text-to-speech (TTS), or others; to run fast inference for reinforcement learning (RL) even when models aren’t yet supported in other libraries. We're excited for Transformers v5 and are super happy to be working with the Hugging Face team!
Transformers, at the core, remains a model architecture toolkit. We aim to have all recent architectures and to be the “source of truth” for model definitions. We’ve been adding between 1 - 3 new models every week for 5 years, shown in the timeline below:
We’re building tooling to help us identify which existing model architecture a new model resembles. This feature uses machine learning to find code similarities between independent modeling files. Going further, we aim to automate the conversion process by opening a draft PR for the model to be integrated into our transformers format. This process reduces manual effort and ensures consistency.
We introduce a significant change to the way we load weights in our models; and with this, we move to quantization being a first-class citizen.
The first focus of the team was on simplicity. Working on transformers, we see the code as the product. We want our model integrations to be clean, so that the ecosystem may depend on our model definitions and understand what’s really happening under the hood, how models differ from each other, and the key features of each new model. Simplicity results in wider standardization, generality, and wider support.
Over the past couple of years, the increasing amount of 0-day support for new model architectures and standardization of attention handling has helped to simplify our support for post-training modern LLMs.
Finally, we’re sunsetting our Flax/TensorFlow support in favor of focusing on PyTorch as the sole backend; however, we're also working with partners in the Jax ecosystem to ensure we have compatibility between our models and this ecosystem.
This growth is powered by the evolution of the field and the now mainstream access to AI. As a leading model-definition library in the ecosystem, we need to continuously evolve and adapt the library to continue being relevant. Reinvention is key for longevity in AI.
We’re fortunate to collaborate with many libraries and apps built on transformers, in no specific order: llama.cpp, MLX, onnxruntime, Jan, LMStudio, vLLM, SGLang, Unsloth, LlamaFactory, dLLM, MaxText, TensorRT, Argmax, among many other friends.
For v5, we wanted to work on several notable aspects: simplicity, training, inference, and production. We detail the work that went into them in this post.
The scale of adoption and the expansion of supported architectures position Transformers as the central model-definition library in the AI ecosystem, enabling faster development, training, and deployment across diverse frameworks.