Datasets
Dataset
tokenwise_lid_all_scripts_dataset
by blue-machines
blue-machines/tokenwiselidallscriptsdataset Word/token-level LID training corpus (tokenlidcombineddataset. csv).
No licensetoken-classification 27 01M < n < 10M rows
published 27 Jul 2026
Preview
The Hugging Face dataset viewer has no preview for this one.
Cite this
No licenseCite
Tags
task_categories:token-classificationlanguage:enlanguage:hilanguage:bnlanguage:gulanguage:knlanguage:mllanguage:mrlanguage:orlanguage:talanguage:telicense:othersize_categories:1M<n<10Mregion:uslanguage-identificationtoken-classificationindiccode-mixedromanized