Certifying Lower Bounds for Risk-Sensitive Reinforcement Learning under Adversarial State Perturbations — Paper

Abstract

Reinforcement learning (RL) agents deployed in real-world environments are often vulnerable to adversarial perturbations in state observations, creating risks in safety-critical applications. Certification methods can improve robustness against adversarial perturbations by providing lower bounds on expected cumulative rewards. Existing certification methods, however, mainly focus on risk-neutral objectives. In this paper, we extend certification methods to risk-sensitive objectives by establishing lower bounds on the exponential utility of cumulative rewards under $l_{p}$-norm-bounded state adversarial perturbations ($1\leq p <\infty$). By introducing a $\phi$-divergence relaxation of the perturbation set, we formulate the risk-sensitive certification problem as a convex optimization and derive its dual to obtain a tractable approximation of the certified lower bound. We further propose an empirical method that improves certified lower bounds by selecting the training risk-aversion parameter $\beta$ independently of the risk level used during evaluation. Experiments on both OpenAI Gym environments and a machine replacement problem show that, compared to risk-neutral training, risk-averse training generally yields policies with higher certified lower bounds, particularly under larger perturbation budgets. Moreover, under both risk-neutral and risk-averse evaluation settings, increasing risk aversion during training leads to non-monotonic certification performance, where certified lower bounds initially improve but eventually decrease due to overly conservative policies.

Authors 3

Tong Li, Saunak Kumar Panda, Yisha Xiang

Specification

Official page: arxiv.org/abs/2609.10866
Source:arXiv (Atom API + RSS)T1observed 41 min agohigh
Arxiv announce type: new
Source:arXiv (Atom API + RSS)T1observed 41 min agohigh
arXiv id: 2609.10866
Source:arXiv (Atom API + RSS)T1observed 41 min agohigh
Categories: cs.LG, math.OC
Source:arXiv (Atom API + RSS)T1observed 41 min agohigh
PDF: arxiv.org/pdf/2609.10866
Source:arXiv (Atom API + RSS)T1observed 41 min agohigh
Primary category: cs.LG
Source:arXiv (Atom API + RSS)T1observed 41 min agohigh
Published: 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 41 min agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

9

Source tiers

T19

Freshest observation

41 min ago

Conflicts

None

No models linked to this paper yet.

Authors: Tong Li, Saunak Kumar Panda, Yisha Xiang

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history · Categories

1 claims · 1 propertiesShow all properties

Categoriescategories1

Claim history for Categories
Value	Valid from → to	Status	Source	Confidence	Extractor
cs.LG, math.OC	11 Sept 2026 21:02 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
New paperPaperCertifying Lower Bounds for Risk-Sensitive Reinforcement Learning under Adversarial State Perturbations
New paper: Certifying Lower Bounds for Risk-Sensitive Reinforcement Learning under Adversarial State Perturbations
arxiv
rss.arxiv.org

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.LG	feed	T1· Official	41 min ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.