Opening article
Hugging Face has released vdr-2b-multi-v1, a multilingual visual document retrieval model that encodes document page screenshots into single dense vectors, eliminating the need for OCR or extraction pipelines. Built on MrLight/dse-qwen2-2b-mrl-v1 and trained on 500k multilingual query‑image pairs across Italian, Spanish, English, French and German, the model achieves faster inference and lower VRAM usage while improving retrieval performance. The training data, sourced from public PDFs and generated with VLMs, was cleaned and filtered to remove low‑quality queries, and hard negatives were mined to strengthen the model.
Introducing vdr-2b-multi-v1 (🤗) , a multilingual embedding model designed for visual document retrieval across multiple languages and domains. This model is designed to encode document page screenshots into dense single-vector representations, this will effectively allow to search and query visually rich multilingual documents without the need for any OCR, data extraction pipelines, chunking...
The vdr-2b-multi-v1 model is based on MrLight/dse-qwen2-2b-mrl-v1 and is trained on an extensive self-made dataset of multilingual query-image pairs. This model is built in collaboration with LlamaIndex and is the next iteration of mcdse-2b-v1 . Our vdr-2b-multi-v1 extends and improves the learning and methods used to train it, resulting in a much more powerful and better model.
Trained on 🇮🇹 Italian, 🇪🇸 Spanish, 🇬🇧 English, 🇫🇷 French and 🇩🇪 German: Together they form a new large, open-source, multilingual training dataset of 500k high-quality samples.
Cross-lingual Retrieval : Substantially better on real-world scenarios. For example, you can search for German documents with Italian queries.
The model is significantly better across all document types, with an average improvement of +2.3%. These retrieval capabilities are essential for real-world use cases, especially in linguistically fragmented continents such as Europe. For example, it enables language-independent searches on complex multilingual sources such as European binding decisions, instruction manuals, financial asset KIDs, pharmaceutical package leaflets and many more...
Low VRAM and Faster Inference : On synthetic Visual Document Retrieval (ViDoRe) benchmarks, our English-only model with 768 image patches performs better than the base model with 2560 image patches. This results in 3x faster inference and much lower VRAM usage.
Matryoshka Representation Learning : You can reduce the vectors size 3x and still keep 98% of the embeddings quality. This allows for notably faster retrieval speeds while reducing storage costs.
Generating embeddings with vdr-2b-multi-v1 is easier than ever with SentenceTransformers and LlamaIndex direct integrations. Get started with just a few lines of code:
The model enables efficient, language‑agnostic search of complex documents, reducing infrastructure costs and expanding AI‑powered retrieval to multilingual contexts.