Opening article
Hugging Face has partnered with the Indian Institute of Science (IISc) and ARTPARK to enhance access to Vaani, an open‑source, multi‑modal, multi‑lingual dataset covering India’s linguistic diversity. The collaboration aims to improve usability of Vaani, which contains over 150,000 hours of speech and 15,000 hours of transcribed text from 1 million speakers across 773 districts. Phase 1 of the dataset, covering 80 districts, has already been released, and Phase 2 is expanding coverage to 100 more districts.
The Indian Institute of Science IISc and ARTPARK partner with Hugging Face to enable developers across the globe to access Vaani , India's most diverse open-source, multi-modal, multi-lingual dataset. Both organisations share a commitment to building inclusive, accessible, and state-of-the-art AI technologies that honor linguistic and cultural diversity.
The partnership between Hugging Face and IISc/ARTPARK aims to increase the accessibility and improve usability of the Vaani dataset, encouraging the development of AI systems that better understand India's diverse languages and cater to the digital needs of its people.
Launched in 2022 by IISc/ARTPARK and Google, Project Vaani is a pioneering initiative aimed at creating an open-source multi-modal dataset that truly represents India's linguistic diversity. This dataset is unique in its geo-centric approach, allowing for the collection of dialects and languages spoken in remote regions rather than focusing solely on mainstream languages.
These AI models can power a wide range of Conversational AI applications. From educational tools to telemedicine platforms, healthcare solutions, voter helplines, media localization, and multilingual smart devices, the Vaani dataset can be a game-changer in real-world scenarios.
The most meaningful contribution you can make is to use the Vaani dataset. Whether building new AI applications, conducting research, or exploring innovative use cases, your engagement helps improve and expand the project.
Vaani targets the collection of over 150,000 hours of speech and 15,000 hours of transcribed text data from 1 million people across all 773 districts, ensuring diversity in language, dialects, and demographics.
The dataset is being built in phases, with Phase 1 covering 80 districts, which has already been open-sourced. Phase 2 is currently underway, expanding the dataset to 100 more districts, further strengthening Vaani's reach and impact across India's diverse linguistic landscape.
The Vaani dataset shows a rich distribution of languages across India's districts, highlighting linguistic diversity at a local level. This information is valuable for researchers, AI developers, and language technology innovators looking to build speech models tailored to specific regions and dialects. To explore the detailed district-wise language distribution, visit: Vaani Dataset on HuggingFace
Providing a large, geographically diverse dataset empowers developers to create AI systems that better serve multilingual and regional communities, addressing gaps in current language models.