Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery — Paper

Abstract

Agentic systems are rapidly moving to production, where they read untrusted inputs, call tools with real permissions, and act autonomously, expanding the security surface beyond chat-only models. Yet standard evaluations remain single-turn and fail to capture multi-step agent vulnerabilities. We present a systematic black-box framework for risk-aware agent evaluation requiring only basic system descriptions. Our approach introduces: (1) a seven-domain taxonomy mapping observable behaviors to risk categories, (2) fully automated SAGE-RT red teaming producing 120 adversarial scenarios per domain, and (3) human-validated evaluation using LLM judges. Empirical validation across two agent architectures (CrewAI and AutoGen) with four base models reveals alarming patterns: 56.25\% average governance risk, 65\% privacy risk in multi-agent configurations, and agent behavior vulnerabilities reaching 85\%. Our black-box approach effectively identifies critical architectural vulnerabilities without privileged access, providing a scalable path toward safer agent deployments.

Authors 5

Divyanshu Kumar, Nitin Aravind Birur, Tanay Baswa, Sahil Agarwal, Prashanth Harshangi

Specification

Official page: arxiv.org/abs/2609.09647
Source:arXiv (Atom API + RSS)T1observed 6 h agohigh
Arxiv announce type: new
Source:arXiv (Atom API + RSS)T1observed 6 h agohigh
arXiv id: 2609.09647
Source:arXiv (Atom API + RSS)T1observed 6 h agohigh
Categories: cs.AI
Source:arXiv (Atom API + RSS)T1observed 6 h agohigh
PDF: arxiv.org/pdf/2609.09647
Source:arXiv (Atom API + RSS)T1observed 6 h agohigh
Primary category: cs.AI
Source:arXiv (Atom API + RSS)T1observed 6 h agohigh
Published: 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 6 h agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

9

Source tiers

T19

Freshest observation

6 h ago

Conflicts

None

No models linked to this paper yet.

Authors: Divyanshu Kumar, Nitin Aravind Birur, Tanay Baswa

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history · Authors

1 claims · 1 propertiesShow all properties

Authorsauthors1

Claim history for Authors
Value	Valid from → to	Status	Source	Confidence	Extractor
Divyanshu Kumar, Nitin Aravind Birur, Tanay Baswa, Sahil Agarwal, Prashanth Harshangi	11 Sept 2026 21:03 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
New paperPaperBlack-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery
New paper: Black-Box Red Teaming of Agentic AI: A Taxonomy-Driven Framework for Automated Risk Discovery
arxiv
rss.arxiv.org

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.AI	feed	T1· Official	5 h ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.