Dataset

last-translation-benchmark

Name: last-translation-benchmark
Creator: zouhar
License: CC-BY-4.0

by zouhar

Last Translation Benchmark Abstract: For scientific progress, we need benchmarks that test the limits of state-of-the-art models, and evaluation methods that inform us about failure cases. Standard benchmarks for machine translation evaluation are often eit…

CC-BY-4.0translation 1 01K < n < 10K rows

published 31 Aug 2026

Open on Hugging Face

Preview

First 5 rows of default / train, from the Hugging Face dataset viewer.

id	source_text	source_lang	target_lang	source_lang_iso	target_lang_iso	translations	verification_rules	linguistics	tags
8	what's the difference between jail and prison?	English	Czech	eng	ces	[{"model":"Lara","translation":"jaký je rozdíl mezi vězením a věznicí?","verified":[false,true]},{"model":"Google Translate","translation":"jaký je rozdíl mezi vězením a vězením?","verified":[false,true]},{"model":"Gemini 2.5 Flash","translation":"Jaký je rozdíl mezi vězením a káznicí?","verified":…	["The two main words that refer to jail/prison should not be entirely identical, such as \"vězením\" a \"věznicí\" which are pure synonyms. However, \"vazební věznicí\" and \"věznicí\" are distinct.","The words should refer to jail/prison, so věznice, káznice, žalář, vazební cela, vazební věznice, …	["Lexical Gradation","Semantic/Lexical"]	["LTBv1"]
9	ما در پیاله عکس رخ یار دیده ایم	Persian	English	fas	eng	[{"model":"Google Translate","translation":"We have seen his face in the photo cup","verified":[false,true,false]},{"model":"Gemini 2.5 Flash","translation":"ما در پیاله عکس رخ یار دیده‌ایم","verified":[true,false,false]},{"model":"Gemma 4","translation":"We have seen the reflection of the beloved'…	["Translation can't contain exactly \"cup\".","The translation should be in English","The translation should contain \"reflection\" or \"picture.\""]	["Poetry","Polysemy","Style Preservation","Semantic/Lexical"]	["LTBv1"]
10	Il fattone si stava facendo una spada nel parco	Italian	English	ita	eng	[{"model":"Lara","translation":"The big guy was making a sword in the park","verified":[false]},{"model":"Google Translate","translation":"The stoner was having a sword in the park","verified":[false]},{"model":"Gemini 2.5 Flash","translation":"The stoner was having a blowjob in the park","verified…	["Should contain 'syringe' or 'drugs', not 'sword'"]	["Slang","Metaphor","Collocation","Semantic/Lexical"]	["LTBv1","LTBv1-eval"]
12	مدیریت بحران منطقه	Farsi	English	fas	eng	[{"model":"Gemini 2.5 Flash","translation":"Crisis Management Region","verified":[true,true,false]},{"model":"Gemma 4","translation":"Regional crisis management","verified":[false,true,true]},{"model":"Llama 4 Maverick","translation":"Crisis management in the region","verified":[true,false,true]},{…	["Translation can't contain \"regional\".","Translation can't contain \"in\" or \"for\".","In the translation, \"management\" should be the main subject."]	["Target Gap: Morph to Words","Collocation","Syntactic","Morphological","Semantic/Lexical"]	["LTBv1"]
13	صورت تو ملک جمال و جمال ملک است	Farsi	English	fas	eng	[{"model":"Gemini 2.5 Flash","translation":"Your face is the land of beauty and beauty is the land.","verified":[false,true]},{"model":"Gemma 4","translation":"Your face is the kingdom of beauty and the beauty of the kingdom.","verified":[true,true]},{"model":"Llama 4 Scout","translation":" face is…	["Translation can't contain exactly \"beauty is\".","Translation has to contain exactly \"Your face\"."]	["Gardenpath","Poetry","Syntactic","Orthography"]	["LTBv1"]

Cite this

CC-BY-4.0textCite

last-translation-benchmark

Preview

Cite this

Tags