IndicTriMix: Developing Language Identification Datasets and Models for Tri-Language Code-Mixing — Paper

Abstract

Language identification in code-mixed text, largely observed in social media, is highly essential when users frequently switch between multiple languages within a single utterance. Accurately identifying the languages of code-mixed tokens becomes an urgent necessity. Traditional language identification models, designed for monolingual text, are not well suited for token-level language identification in code-mixed settings. We formulate the task as a sequence labeling problem and fine-tune contextual transformer-based models MuRIL and XLM-RoBERTa best suited for Indian languages. We evaluate these systems on three different data configurations (Hindi, Gujarati, and Bengali) to predict language labels for individual tokens. We release a benchmark for language identification in code-mixed tokens with manually annotated test sets. We propose two approaches of code-mixed generation using parallel sentences of three languages. The trained models demonstrate the effectiveness of contextual embeddings for token-level language identification in multilingual social media text. For reproducibility and to facilitate future research, we publicly release our fine-tuned models.

Authors 5

Pruthwik Mishra, Rudra Trivedi, Avi Patel, Ashok Urlana, Shrikant Malviya

Specification

Official page: arxiv.org/abs/2609.11851
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
Arxiv announce type: new
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
arXiv id: 2609.11851
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
Categories: cs.CL
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
PDF: arxiv.org/pdf/2609.11851
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
Primary category: cs.CL
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh
Published: 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 7 h agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

9

Source tiers

T19

Freshest observation

7 h ago

Conflicts

None

No models linked to this paper yet.

Authors: Pruthwik Mishra, Rudra Trivedi, Avi Patel

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history · arXiv id

1 claims · 1 propertiesShow all properties

arXiv idarxiv_id1

Claim history for arXiv id
Value	Valid from → to	Status	Source	Confidence	Extractor
2609.11851	11 Sept 2026 21:03 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
New paperPaperIndicTriMix: Developing Language Identification Datasets and Models for Tri-Language Code-Mixing
New paper: IndicTriMix: Developing Language Identification Datasets and Models for Tri-Language Code-Mixing
arxiv
rss.arxiv.org

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.CL	feed	T1· Official	5 h ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.