Empirical Evaluation of Membership Inference Attacks on NLP Text Classifiers: A Baseline Study on SST-2 — Paper

Abstract

Membership inference attacks (MIAs) try to determine whether a specific record was used to train a model, a privacy risk that matters in natural language processing (NLP), where training data can contain sensitive user text. This paper presents a controlled benchmark of membership inference vulnerability for text classification on the GLUE SST-2 sentiment dataset. A TF-IDF + Logistic Regression pipeline and a fine-tuned DistilBERT classifier are compared under a loss-threshold MIA, with utility measured by development accuracy and macro F1. DistilBERT reached 0.9466 accuracy and 0.9460 macro F1 against 0.8756 and 0.8727 for Logistic Regression, yet both models leaked membership signal (Attack AUC 0.5615 and 0.5800, respectively). Two mitigations were tested. Stronger regularization reduced leakage for Logistic Regression at a visible utility cost, whereas fine-tuning DistilBERT for 2 epochs instead of 3 reduced leakage with negligible accuracy loss. Lightweight training adjustments can improve the privacy-utility trade-off without complex defenses.

Authors 2

William Novak (Minot State University), Muhammad Abusaqer (Minot State University)

Specification

Official page: arxiv.org/abs/2609.10935
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
Arxiv announce type: cross
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
arXiv id: 2609.10935
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
Categories: cs.CR, cs.CL, cs.LG
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
PDF: arxiv.org/pdf/2609.10935
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
Primary category: cs.CR
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
Published: 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

9

Source tiers

T19

Freshest observation

2 h ago

Conflicts

None

No models linked to this paper yet.

Authors: William Novak (Minot State University), Muhammad Abusaqer (Minot State University)

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history · PDF

1 claims · 1 propertiesShow all properties

PDFpdf_url1

Claim history for PDF
Value	Valid from → to	Status	Source	Confidence	Extractor
https://arxiv.org/pdf/2609.10935	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
New paperPaperEmpirical Evaluation of Membership Inference Attacks on NLP Text Classifiers: A Baseline Study on SST-2
New paper: Empirical Evaluation of Membership Inference Attacks on NLP Text Classifiers: A Baseline Study on SST-2
arxiv
rss.arxiv.org

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.LG	feed	T1· Official	14 min ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.