Certifying Lower Bounds for Risk-Sensitive Reinforcement Learning under Adversarial State Perturbations
Updated 41 min ago · first seen 11 Sept 2026
paper_01M294FNTTK8HPE5RPTH3G028X
- Published
- 11 Sept 2026
- T1 · 41 min ago
- arXiv
- 2609.10866
- T1 · 41 min ago
- Category
- cs.LG
- T1 · 41 min ago
Abstract
Reinforcement learning (RL) agents deployed in real-world environments are often vulnerable to adversarial perturbations in state observations, creating risks in safety-critical applications. Certification methods can improve robustness against adversarial perturbations by providing lower bounds on expected cumulative rewards. Existing certification methods, however, mainly focus on risk-neutral objectives. In this paper, we extend certification methods to risk-sensitive objectives by establishing lower bounds on the exponential utility of cumulative rewards under $l_{p}$-norm-bounded state adversarial perturbations ($1\leq p <\infty$). By introducing a $\phi$-divergence relaxation of the perturbation set, we formulate the risk-sensitive certification problem as a convex optimization and derive its dual to obtain a tractable approximation of the certified lower bound. We further propose an empirical method that improves certified lower bounds by selecting the training risk-aversion parameter $\beta$ independently of the risk level used during evaluation. Experiments on both OpenAI Gym environments and a machine replacement problem show that, compared to risk-neutral training, risk-averse training generally yields policies with higher certified lower bounds, particularly under larger perturbation budgets. Moreover, under both risk-neutral and risk-averse evaluation settings, increasing risk aversion during training leads to non-monotonic certification performance, where certified lower bounds initially improve but eventually decrease due to overly conservative policies.
Authors 3
Tong Li, Saunak Kumar Panda, Yisha Xiang
Specification
- Official page
Source:arXiv (Atom API + RSS)T1observed 41 min agohigh
- Arxiv announce type
- new
Source:arXiv (Atom API + RSS)T1observed 41 min agohigh
- arXiv id
- 2609.10866
Source:arXiv (Atom API + RSS)T1observed 41 min agohigh
- Categories
- cs.LG, math.OC
Source:arXiv (Atom API + RSS)T1observed 41 min agohigh
Source:arXiv (Atom API + RSS)T1observed 41 min agohigh
- Primary category
- cs.LG
Source:arXiv (Atom API + RSS)T1observed 41 min agohigh
- Published
- 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 41 min agohigh
Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →
Provenance
Attributed facts
9
Source tiers
T19
Freshest observation
41 min ago
Conflicts
None
No models linked to this paper yet.
- Authors
- Tong Li, Saunak Kumar Panda, Yisha Xiang
As of
Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.
Claim history · Abstract
Abstractabstract1
| Value | Valid from → to | Status | Source | Confidence | Extractor |
|---|---|---|---|---|---|
| Reinforcement learning (RL) agents deployed in real-world environments are often vulnerable to adversarial perturbations in state observations, creating risks in safety-critical applications. Certification methods can improve robustness against adversarial perturbations by providing lower bounds on expected cumulative rewards. Existing certification methods, however, mainly focus on risk-neutral objectives. In this paper, we extend certification methods to risk-sensitive objectives by establishing lower bounds on the exponential utility of cumulative rewards under $l_{p}$-norm-bounded state adversarial perturbations ($1\leq p <\infty$). By introducing a $\phi$-divergence relaxation of the perturbation set, we formulate the risk-sensitive certification problem as a convex optimization and derive its dual to obtain a tractable approximation of the certified lower bound. We further propose an empirical method that improves certified lower bounds by selecting the training risk-aversion parameter $\beta$ independently of the risk level used during evaluation. Experiments on both OpenAI Gym environments and a machine replacement problem show that, compared to risk-neutral training, risk-averse training generally yields policies with higher certified lower bounds, particularly under larger perturbation budgets. Moreover, under both risk-neutral and risk-averse evaluation settings, increasing risk aversion during training leads to non-monotonic certification performance, where certified lower bounds initially improve but eventually decrease due to overly conservative policies. | → current | current | arXiv (Atom API + RSS)T1 | high | deterministic |
Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →
- New paperPaperCertifying Lower Bounds for Risk-Sensitive Reinforcement Learning under Adversarial State Perturbations
New paper: Certifying Lower Bounds for Risk-Sensitive Reinforcement Learning under Adversarial State Perturbations
arxiv
| Source | Document | Type | Tier | Last observed | Snapshots |
|---|---|---|---|---|---|
| arXiv (Atom API + RSS) | rss.arxiv.org/rss/cs.LG | feed | T1· Official | 41 min ago | 1 |
Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.