Multilingual in Name Only? Cultural and Linguistic Weaknesses of LLMs in Urdu — Paper

Abstract

Multilingual large language models (LLMs) are increasingly used for open-ended text generation, yet their behaviour in low-resource languages remains poorly understood. In this work, we question how correct and reliable is the generation of multilingual LLMs when used for the task of story generation. We consider Urdu language as a representative low-resource language. We generate Urdu-Stories, a corpus of 93 stories generated using three contemporary LLMs (GPT-5.1, Qwen-3-Max, DeepSeek-3.1). We manually annotate the errors present in them under a nine-label linguistic, semantic, and cultural taxonomy. Our notable findings suggest that LLMs often make basic errors of grammar and semantics. The stories lack coherence, have unnatural repetition and show pervasive cultural shallowness. We further show using few-shot prompting that the cultural and context errors largely remain unresolved. Our findings highlight the limitations of current LLMs as a reliable source of content generation and information retrieval for low-resource languages.

Authors 4

Farah Adeeba, Abdul Rafae Khan, Rajesh Bhatt, Hassan Sajjad

Specification

Official page: arxiv.org/abs/2609.10758
Source:arXiv (Atom API + RSS)T1observed 1 h agohigh
Arxiv announce type: new
Source:arXiv (Atom API + RSS)T1observed 1 h agohigh
arXiv id: 2609.10758
Source:arXiv (Atom API + RSS)T1observed 1 h agohigh
Categories: cs.CL, cs.AI, cs.LG
Source:arXiv (Atom API + RSS)T1observed 1 h agohigh
PDF: arxiv.org/pdf/2609.10758
Source:arXiv (Atom API + RSS)T1observed 1 h agohigh
Primary category: cs.CL
Source:arXiv (Atom API + RSS)T1observed 1 h agohigh
Published: 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 1 h agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

9

Source tiers

T19

Freshest observation

1 h ago

Conflicts

None

No models linked to this paper yet.

Authors: Farah Adeeba, Abdul Rafae Khan, Rajesh Bhatt

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history · Abstract

1 claims · 1 propertiesShow all properties

Abstractabstract1

Claim history for Abstract
Value	Valid from → to	Status	Source	Confidence	Extractor
Multilingual large language models (LLMs) are increasingly used for open-ended text generation, yet their behaviour in low-resource languages remains poorly understood. In this work, we question how correct and reliable is the generation of multilingual LLMs when used for the task of story generation. We consider Urdu language as a representative low-resource language. We generate Urdu-Stories, a corpus of 93 stories generated using three contemporary LLMs (GPT-5.1, Qwen-3-Max, DeepSeek-3.1). We manually annotate the errors present in them under a nine-label linguistic, semantic, and cultural taxonomy. Our notable findings suggest that LLMs often make basic errors of grammar and semantics. The stories lack coherence, have unnatural repetition and show pervasive cultural shallowness. We further show using few-shot prompting that the cultural and context errors largely remain unresolved. Our findings highlight the limitations of current LLMs as a reliable source of content generation and information retrieval for low-resource languages.	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
Property changedPaperMultilingual in Name Only? Cultural and Linguistic Weaknesses of LLMs in Urdu
Multilingual in Name Only? Cultural and Linguistic Weaknesses of LLMs in Urdu: arxiv announce type changed from cross to new
Arxiv announce typecross→newarxiv
rss.arxiv.org
11 Sept 2026
New paperPaperMultilingual in Name Only? Cultural and Linguistic Weaknesses of LLMs in Urdu
New paper: Multilingual in Name Only? Cultural and Linguistic Weaknesses of LLMs in Urdu
arxiv
rss.arxiv.org

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.CL	feed	T1· Official	1 h ago	1
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.LG	feed	T1· Official	1 h ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.