Timm ❤️ Transformers: Use any timm model with transformers

Get lightning-fast inference, quick quantization, torch.compile boosts, and effortless fine-tuning for any timm model—all within the friendly 🤗 transformers ecosystem.

Enter TimmWrapper —a simple, yet powerful tool that unlocks this potential.

To follow along with this blog post, install the latest version of transformers and timm by running:

Let's see how to use a transformers pipeline with the MobileNetV4 . This architecture does not have a native transformers implementation, but can be easily used from timm :

Fine-tuning a timm model with the Trainer API from 🤗 transformers is straightforward and highly flexible . You can fine-tune your model on custom datasets using the Trainer class, which handles the training loop, logging, and evaluation. Additionally, you can fine-tune using LoRA (Low-Rank Adaptation) to train efficiently with fewer parameters.

LoRA (Low-Rank Adaptation) allows you to fine-tune large models efficiently by training only a few additional parameters, rather than the full set of model weights. This makes fine-tuning faster, and allows the use of consumer hardware. You can fine-tune a timm model using LoRA with the PEFT library .

LoRA is just one example of efficient adapter-based fine-tuning methods you can apply to timm models. The integration of timm with the 🤗 ecosystem opens up a wide variety of parameter-efficient fine-tuning (PEFT) techniques, allowing you to choose the method that best fits your use case.

With pre-trained models covering a wide range of architectures, timm simplifies the workflow for computer vision practitioners.

What's remarkable about this approach is that it mirrors the exact workflow used for native transformers models, maintaining consistency across different model types.

timm 's integration with transformers opens new doors for leveraging state-of-the-art vision models with minimal effort. Whether you're looking to fine-tune, quantize, or simply run inference, this integration provides a unified API to streamline your workflow.

Check out the full repository for all code examples and notebooks: 🔗 TimmWrapper Examples

The PyTorch Image Models ( timm ) library offers a rich collection of state-of-the-art computer vision models, along with useful layers, utilities, optimizers, and data augmentations. With more than 32K GitHub stars and more than 200K daily downloads at the time of writing, it's a go-to resource for image classification and feature extraction for object detection, segmentation, image search, and other downstream tasks.

While 🤗 transformers supports several vision models, timm offers an even broader collection, including many mobile-friendly and efficient models not available in transformers.