ଓOpenOdia

Browser

Odia datasets.

Live from Hugging Face — every dataset tagged for Odia. Parallel corpora, speech, classification, instruction-tuning, each with its size, license, and a citation.

★ Featuredrotates every Monday
Text Generation

oscar

@oscar-corpus

The Open Super-large Crawled ALMAnaCH coRpus is a huge multilingual corpus obtained by language classification and filtering of the Common Crawl corpus using the goclassy architecture.

524 208
Text Generation

cc100

@statmt

This corpus is an attempt to recreate the dataset used for training XLM-R. This corpus comprises of monolingual data for 100+ languages and also includes data for romanized languages (indicated by rom).

2.0k 107
Also worth a look
Translation

Pralekha

624 15Other

indian-legal-supervised-fine-tuning-data

89 5Summarization

varta

991 3
Text Generation

wikipedia

@wikimedia

Dataset Card for Wikimedia Wikipedia Dataset Summary Wikipedia dataset containing cleaned articles of all languages. The dataset is built from the Wikipedia dumps with one subset per language, each containing a single train split.

10M < n < 100M rows 239.9k 1.4k
CC-BY-SA-3.0textCite
Text Generation

wikipedia

@legacy-datasets

Wikipedia dataset containing cleaned articles of all languages. The datasets are built from the Wikipedia dump with one split per language.

n < 1K rows 100.7k 662
CC-BY-SA-3.0Cite
Question Answering

belebele

@facebook

The Belebele Benchmark for Massively Multilingual NLU Evaluation Belebele is a multiple-choice machine reading comprehension (MRC) dataset spanning 122 language variants. This dataset enables the evaluation of mono- and multi-lingual models in high-, medium…

100K < n < 1M rows 46.9k 132
CC-BY-SA-4.0textCite
Automatic Speech Recognition

common_voice

@legacy-datasets

Common Voice is Mozilla's initiative to help teach machines how real people speak. The dataset currently consists of 7,335 validated hours of speech in 60 languages, but we’re always adding more voices and languages.

100K < n < 1M rows 41.5k 148
CC0-1.0Cite
Other

Global-MMLU-Lite

@CohereLabs

Releases: Version 3. 0 (May 2026): GMMLU Lite 3.

10K < n < 100K rows 41.1k 43
Apache-2.0textCite
Other

xP3

@bigscience

xP3 (Crosslingual Public Pool of Prompts) is a collection of prompts & datasets across 46 of languages & 16 NLP tasks. It is used for the training of BLOOMZ and mT0, multilingual language models capable of following human instructions in dozens of languages…

100M < n < 1B rows 34.1k 113
Apache-2.0Cite
Text To Speech

yodas2_sidon

@sarulab-speech

YODAS2-Sidon Overview This dataset is a cleansed version of YODAS-2 with Sidon speech restoration mode for Speech Synthesis and Spoken Language Modeling. YODAS-2 is a massive, multilingual YouTube-derived dataset.

1M < n < 10M rows 26.8k 65
CC-BY-3.0audiotextCite
Automatic Speech Recognition

common_voice_22_0

@fsicoli

Dataset Card for Common Voice Corpus 22. 0 This dataset is an unofficial version of the Mozilla Common Voice Corpus 22.

100B < n < 1T rows 24.4k 19
CC0-1.0Cite
Other

xP3all

@bigscience

xP3 (Crosslingual Public Pool of Prompts) is a collection of prompts & datasets across 46 of languages & 16 NLP tasks. It is used for the training of BLOOMZ and mT0, multilingual language models capable of following human instructions in dozens of languages…

10M < n < 100M rows 19.7k 32
Apache-2.0textCite
Token Classification

wikiann

@unimelb-nlp

Dataset Card for WikiANN Dataset Summary WikiANN (sometimes called PAN-X) is a multilingual named entity recognition dataset consisting of Wikipedia articles annotated with LOC (location), PER (person), and ORG (organisation) tags in the IOB2 format. This v…

1M < n < 10M rows 19.3k 124
No licensetextCite
Automatic Speech Recognition

Vaani

@ARTPARK-IISc

VAANI is an India-representative multi-modal multi-lingual dataset. The current version (phase 1- 80 districts, phase 2- 85 districts) contains ~31,255 hours of spontaenous,image-prompted speech by 156K speakers across 165 districts, talking about 288K imag…

10M < n < 100M rows 19.2k 144
CC-BY-4.0Cite
Text Generation

sangraha

@ai4bharat

Sangraha Sangraha is the largest high-quality, cleaned Indic language pretraining data containing 251B tokens summed up over 22 languages, extracted from curated sources, existing multilingual corpora and large scale translations.

100M < n < 1B rows 19.1k 83
CC-BY-4.0textCite
Translation

opus-100

@Helsinki-NLP

Dataset Card for OPUS-100 Dataset Summary OPUS-100 is an English-centric multilingual corpus covering 100 languages. OPUS-100 is English-centric, meaning that all training pairs include English on either the source or target side.

10M < n < 100M rows 17.9k 243
No licensetextCite
Other

Complete_Data_Source_100K_HOURS

@RidheshBhati

Multi-Language Audio Collection (100K Hours) This repository is physically reorganized for Absolute 100% Data Visibility. 🏗️ Global Consolidator Select your language subset to listen to high-quality waveform audio.

1M < n < 10M rows 16.8k 4
No licenseaudiotextCite
Text Generation

IITB-IndicMonoDoc

@cfilt

IITB Document level Monolingual Corpora for Indian languages. 22 scheduled languages of India + English (1) Assamese, (2) Bengali, (3) Gujarati, (4) Hindi, (5) Kannada, (6) Kashmiri, (7) Konkani, (8) Malayalam, (9) Manipuri, (10) Marathi, (11) Nepali, (12) …

10B < n < 100B rows 12.6k 11
CC-BY-4.0Cite
Text Generation

CulturaX

@uonlp

CulturaX Cleaned, Enormous, and Public: The Multilingual Fuel to Democratize Large Language Models for 167 Languages Dataset Summary We present CulturaX, a substantial multilingual dataset with 6. 3 trillion tokens in 167 languages, tailored for large langu…

1B < n < 10B rows 12.6k 671
No licensetextCite
Other

VoxCommunis

@pacscilab

VoxCommunis Corpus The VoxCommunis Corpus is a phonetic corpus derived from the Mozilla Common Voice Corpus. Corresponding audio files and corpus metadata can be downloaded from Mozilla Common Voice, or from one of several Hugging Face repositories for the …

n < 1K rows 12.0k 9
CC0-1.0Cite
Other

xP3mt

@bigscience

xP3 (Crosslingual Public Pool of Prompts) is a collection of prompts & datasets across 46 of languages & 16 NLP tasks. It is used for the training of BLOOMZ and mT0, multilingual language models capable of following human instructions in dozens of languages…

10M < n < 100M rows 11.6k 26
Apache-2.0textCite
Automatic Speech Recognition

common_voice_17_0

@fsicoli

Dataset Card for Common Voice Corpus 17. 0 This dataset is an unofficial version of the Mozilla Common Voice Corpus 17.

100B < n < 1T rows 11.4k 17
CC0-1.0Cite
Automatic Speech Recognition

common_voice_15_0

@fsicoli

Dataset Card for Common Voice Corpus 15. 0 This dataset is an unofficial version of the Mozilla Common Voice Corpus 15.

100B < n < 1T rows 9.1k 6
No licenseCite
Question Answering

Updesh_beta

@microsoft

📢 Updesh: Synthetic Multilingual Instruction Tuning Dataset for 13 Indic Languages NOTE: This is an initial $\beta$-release. We plan to release subsequent versions of Updesh with expanded coverage and enhanced quality control.

1M < n < 10M rows 9.0k 15
No licensetextCite
Text Classification

ccnews

@stanford-oval

This dataset is the result of processing all WARC files in the CCNews Corpus, from the beginning (2016) to June of 2024. The data has been cleaned and deduplicated, and language of articles have been detected and added.

100M < n < 1B rows 8.4k 36
No licenseimagetextCite
Text Generation

wikipedia-monthly

@omarkamali

🚀 Wikipedia Monthly Last updated: March 14, 2026, 21:06 UTC This repository provides monthly, multilingual dumps of Wikipedia, processed and prepared for easy use in NLP projects. 📊 Current Statistics Metric Current Export (March 2026) All Exports (Total)…

100M < n < 1B rows 8.0k 79
CC-BY-SA-4.0textCite
Text Generation

Wikinium

@Rikunarita-ORG

Wikinium Wikinium is alive. The Living Wikipedia Dataset Wikinium is a continuously evolving Wikipedia dataset optimized for Large Language Model pretraining.

7.7k 7
CC-BY-SA-4.0Cite
Text To Speech

indicvoices_r

@ai4bharat

IndicVoices-R: Multilingual, Multi-Speaker Speech Corpus for Indian TTS Dataset Summary IndicVoices-R (IV-R) is the largest multilingual Indian text-to-speech (TTS) dataset derived from an automatic speech recognition (ASR) dataset. It contains 1,704 hours …

100K < n < 1M rows 7.7k 35
CC-BY-4.0audiotextCite
Other

fineweb-2-sentence-split

@mimir-lcm

Fineweb 2 split into sentences. Instances per languages were sampled by us to balance the data w.

100M < n < 1B rows 6.8k 0
No licensetextCite
Automatic Speech Recognition

common_voice_16_0

@fsicoli

Dataset Card for Common Voice Corpus 16. 0 This dataset is an unofficial version of the Mozilla Common Voice Corpus 16.

100B < n < 1T rows 5.2k 4
CC0-1.0Cite
Audio Classification

ml_spoken_words

@MLCommons

Multilingual Spoken Words Corpus is a large and growing audio dataset of spoken words in 50 languages collectively spoken by over 5 billion people, for academic research and commercial applications in keyword spotting and spoken term search, licensed under …

10M < n < 100M rows 5.1k 37
CC-BY-4.0Cite
Image To Text

NayanaOCR_Corpus_2025

@Cognitive-Lab

🪷 NayanaOCR Corpus 2025 A 1M-page, 22-language fully-parallel synthetic OCR + VQA corpus for document-centric vision-language models — every page rendered in every language. NayanaOCR Corpus 2025 is one of the largest open-source multilingual, multi-task d…

1M < n < 10M rows 5.0k 18
CC-BY-NC-4.0imagetextCite
Audio Classification

LRLspoof

@lab260

LRLspoof [! IMPORTANT] Official dataset for our INTERSPEECH 2026 paper "When Spoof Detectors Travel: Evaluation Across 66 Languages in the Low-Resource Language Spoofing Corpus" (arXiv:2603.

1M < n < 10M rows 5.0k 7
MITCite

Showing 30 of 466 datasets