ଓOpenOdia

Browser

Odia datasets.

Live from Hugging Face — every dataset tagged for Odia. Parallel corpora, speech, classification, instruction-tuning, each with its size, license, and a citation.

★ Featuredrotates every Monday
Question Answering

IndicGenBench_xorqa_in

@google

Dataset Card for Dataset Name This repository contains the XorQA-IN dataset released as a part of the paper "IndicGenBench: A Multilingual Benchmark to Evaluate Generation Capabilities of LLMs on Indic Languages" Pape…

519 4
Question Answering

BharatBBQ

@aditya20t

BharatBBQ A Multilingual Bias Benchmark for Question Answering in the Indian Context Published in the Transactions of the Association for Computational Linguistics (TACL), Vol. 13, and presented at EMNLP 2025 in Suzhou, China.

283 4
Also worth a look
Text Generation

guanaco-llama2-200

58 4Text Retrieval

IndicMSMARCO

214 3Summarization

PMIndiaSum

329 3
Text Generation

wikipedia

@wikimedia

Dataset Card for Wikimedia Wikipedia Dataset Summary Wikipedia dataset containing cleaned articles of all languages. The dataset is built from the Wikipedia dumps with one subset per language, each containing a single train split.

10M < n < 100M rows 295.2k 1.7k
CC-BY-SA-3.0textCite
Text Generation

wikipedia

@legacy-datasets

Wikipedia dataset containing cleaned articles of all languages. The datasets are built from the Wikipedia dump with one split per language.

n < 1K rows 59.6k 684
CC-BY-SA-3.0Cite
Other

xP3

@bigscience

xP3 (Crosslingual Public Pool of Prompts) is a collection of prompts & datasets across 46 of languages & 16 NLP tasks. It is used for the training of BLOOMZ and mT0, multilingual language models capable of following human instructions in dozens of languages…

100M < n < 1B rows 49.4k 114
Apache-2.0Cite
Question Answering

belebele

@facebook

The Belebele Benchmark for Massively Multilingual NLU Evaluation Belebele is a multiple-choice machine reading comprehension (MRC) dataset spanning 122 language variants. This dataset enables the evaluation of mono- and multi-lingual models in high-, medium…

100K < n < 1M rows 41.4k 142
CC-BY-SA-4.0textCite
Text Generation

sangraha

@ai4bharat

Sangraha Sangraha is the largest high-quality, cleaned Indic language pretraining data containing 251B tokens summed up over 22 languages, extracted from curated sources, existing multilingual corpora and large scale translations.

100M < n < 1B rows 38.2k 85
CC-BY-4.0textCite
Token Classification

wikiann

@unimelb-nlp

Dataset Card for WikiANN Dataset Summary WikiANN (sometimes called PAN-X) is a multilingual named entity recognition dataset consisting of Wikipedia articles annotated with LOC (location), PER (person), and ORG (organisation) tags in the IOB2 format. This v…

1M < n < 10M rows 37.4k 126
No licensetextCite
Automatic Speech Recognition

common_voice_16_0

@fsicoli

Dataset Card for Common Voice Corpus 16. 0 This dataset is an unofficial version of the Mozilla Common Voice Corpus 16.

100B < n < 1T rows 26.0k 4
CC0-1.0Cite
Text Generation

CulturaX

@uonlp

CulturaX Cleaned, Enormous, and Public: The Multilingual Fuel to Democratize Large Language Models for 167 Languages Dataset Summary We present CulturaX, a substantial multilingual dataset with 6. 3 trillion tokens in 167 languages, tailored for large langu…

1B < n < 10B rows 25.4k 701
No licensetextCite
Translation

opus-100

@Helsinki-NLP

Dataset Card for OPUS-100 Dataset Summary OPUS-100 is an English-centric multilingual corpus covering 100 languages. OPUS-100 is English-centric, meaning that all training pairs include English on either the source or target side.

10M < n < 100M rows 24.9k 253
No licensetextCite
Automatic Speech Recognition

common_voice_15_0

@fsicoli

Dataset Card for Common Voice Corpus 15. 0 This dataset is an unofficial version of the Mozilla Common Voice Corpus 15.

100B < n < 1T rows 23.7k 6
No licenseCite
Automatic Speech Recognition

Vaani

@ARTPARK-IISc

VAANI is an India-representative multi-modal multi-lingual dataset. The current version (phase 1- 80 districts, phase 2- 85 districts) contains ~31278 hours of spontaenous,image-prompted speech by 156K speakers across 165 districts, talking about 288K image…

1M < n < 10M rows 21.4k 165
CC-BY-4.0audioimagetextCite
Automatic Speech Recognition

bolAIndia

@kapturecx

bolAIndia A growing atlas of India’s spoken voices India does not speak in a single voice. It speaks through a vast and vibrant mosaic of languages, regional accents, dialects, and everyday expressions—across cities, towns, and rural communities.

10M < n < 100M rows 18.7k 4
No licenseaudiodocumenttextCite
Automatic Speech Recognition

common_voice_22_0

@fsicoli

Dataset Card for Common Voice Corpus 22. 0 This dataset is an unofficial version of the Mozilla Common Voice Corpus 22.

100B < n < 1T rows 17.5k 20
CC0-1.0Cite
Other

xP3all

@bigscience

xP3 (Crosslingual Public Pool of Prompts) is a collection of prompts & datasets across 46 of languages & 16 NLP tasks. It is used for the training of BLOOMZ and mT0, multilingual language models capable of following human instructions in dozens of languages…

10M < n < 100M rows 14.9k 33
Apache-2.0textCite
Other

Global-MMLU-Lite

@CohereLabs

Releases: Version 3. 0 (May 2026): GMMLU Lite 3.

10K < n < 100K rows 14.4k 45
Apache-2.0textCite
Other

Complete_Data_Source_100K_HOURS

@RidheshBhati

Multi-Language Audio Collection (100K Hours) This repository is physically reorganized for Absolute 100% Data Visibility. 🏗️ Global Consolidator Select your language subset to listen to high-quality waveform audio.

1M < n < 10M rows 12.0k 4
No licenseaudiotextCite
Other

OPUS_GlobalVoices

@wecover

10M < n < 100M rows 11.9k 0
No licensetextCite
Text Generation

flores_101

@gsarti

One of the biggest challenges hindering progress in low-resource and multilingual machine translation is the lack of good evaluation benchmarks. Current evaluation benchmarks either lack good coverage of low-resource languages, consider only restricted doma…

100K < n < 1M rows 11.8k 34
CC-BY-SA-4.0tabulartextCite
Automatic Speech Recognition

common_voice_17_0

@fsicoli

Dataset Card for Common Voice Corpus 17. 0 This dataset is an unofficial version of the Mozilla Common Voice Corpus 17.

100B < n < 1T rows 11.2k 19
CC0-1.0Cite
Text Generation

oscar-small

@nthngdy

The Open Super-large Crawled ALMAnaCH coRpus is a huge multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the goclassy architecture.

10M < n < 100M rows 10.9k 17
CC0-1.0textCite
Text Generation

wikipedia-monthly

@omarkamali

🚀 Wikipedia Monthly Last updated: October 16, 2026, 09:20 UTC This repository provides monthly, multilingual dumps of Wikipedia, processed and prepared for easy use in NLP projects. 📊 Current Statistics Metric Current Export (October 2026) All Exports (To…

9.7k 84
CC-BY-SA-4.0Cite
Other

xP3mt

@bigscience

xP3 (Crosslingual Public Pool of Prompts) is a collection of prompts & datasets across 46 of languages & 16 NLP tasks. It is used for the training of BLOOMZ and mT0, multilingual language models capable of following human instructions in dozens of languages…

10M < n < 100M rows 8.8k 27
Apache-2.0textCite
Audio Classification

ml_spoken_words

@MLCommons

Multilingual Spoken Words Corpus is a large and growing audio dataset of spoken words in 50 languages collectively spoken by over 5 billion people, for academic research and commercial applications in keyword spotting and spoken term search, licensed under …

10M < n < 100M rows 8.7k 37
CC-BY-4.0Cite
Text Classification

ccnews

@stanford-oval

This dataset is the result of processing all WARC files in the CCNews Corpus, from the beginning (2016) to June of 2024. The data has been cleaned and deduplicated, and language of articles have been detected and added.

100M < n < 1B rows 8.3k 37
No licenseimagetextCite
Text To Speech

Pretraining-V1

@projectkaira

Indic TTS Unified v1 A large-scale, unified collection of speech data for text-to-speech (TTS) and speech research. This dataset consolidates 17 distinct source datasets into a single, schema-normalized resource covering Indian / South Asian languages, plus…

10M < n < 100M rows 8.2k 0
CC-BY-4.0audiotextCite
Other

STT_MODEL

@RidheshBhati

Multilingual STT Dataset Audio and transcript pairs for 50 languages. Each language is a Dataset Viewer configuration with train, validation, and test splits.

1M < n < 10M rows 7.7k 1
No licenseaudiotextCite
Text To Speech

indicvoices_r

@ai4bharat

IndicVoices-R: Multilingual, Multi-Speaker Speech Corpus for Indian TTS Dataset Summary IndicVoices-R (IV-R) is the largest multilingual Indian text-to-speech (TTS) dataset derived from an automatic speech recognition (ASR) dataset. It contains 1,704 hours …

100K < n < 1M rows 6.6k 48
CC-BY-4.0audiotextCite
Token Classification

universal_dependencies

@universal-dependencies

Dataset Card (v2. 0) for Universal Dependencies Treebank Version 2.

1M < n < 10M rows 6.3k 8
No licensetextCite
Image To Text

NayanaOCR_Corpus_2025

@Cognitive-Lab

🪷 NayanaOCR Corpus 2025 A 1M-page, 22-language fully-parallel synthetic OCR + VQA corpus for document-centric vision-language models — every page rendered in every language. NayanaOCR Corpus 2025 is one of the largest open-source multilingual, multi-task d…

1M < n < 10M rows 5.9k 18
CC-BY-NC-4.0imagetextCite
Text Generation

IITB-IndicMonoDoc

@cfilt

IITB Document level Monolingual Corpora for Indian languages. 22 scheduled languages of India + English (1) Assamese, (2) Bengali, (3) Gujarati, (4) Hindi, (5) Kannada, (6) Kashmiri, (7) Konkani, (8) Malayalam, (9) Manipuri, (10) Marathi, (11) Nepali, (12) …

10B < n < 100B rows 5.7k 11
CC-BY-4.0Cite

Showing 30 of 533 datasets