Reinforcement Learning with Temporal-Logic-Based Causal Diagrams — Paper

Abstract

We study a class of reinforcement learning (RL) tasks where the objective of the agent is to accomplish temporally extended goals. In this setting, a common approach is to represent the tasks as deterministic finite automata (DFA) and integrate them into the state-space for RL algorithms. However, while these machines model the reward function, they often overlook the causal knowledge about the environment. To address this limitation, we propose the Temporal-Logic-based Causal Diagram (TL-CD) in RL, which captures the temporal causal relationships between different properties of the environment. We exploit the TL-CD to devise an RL algorithm in which an agent requires significantly less exploration of the environment. To this end, based on a TL-CD and a task DFA, we identify configurations where the agent can determine the expected rewards early during an exploration. Through a series of case studies, we demonstrate the benefits of using TL-CDs, particularly the faster convergence of the algorithm to an optimal policy due to reduced exploration of the environment.

Authors 8

Yash Paliwal, Rajarshi Roy, Jean-Rapha\"el Gaglione, Nasim Baharisangari, Daniel Neider, Xiaoming Duan, Ufuk Topcu, Zhe Xu

Specification

Official page: arxiv.org/abs/2306.13732
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
Arxiv announce type: replace
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
arXiv id: 2306.13732
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
Categories: cs.AI, cs.FL
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
PDF: arxiv.org/pdf/2306.13732
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
Primary category: cs.AI
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh
Published: 11 Sept 2026
Source:arXiv (Atom API + RSS)T1observed 2 h agohigh

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance

Attributed facts

9

Source tiers

T19

Freshest observation

2 h ago

Conflicts

None

No models linked to this paper yet.

Authors: Yash Paliwal, Rajarshi Roy, Jean-Rapha\"el Gaglione

As of

Rewind the record: see this entity's attributes exactly as AI Atlas knew them on a given day.

Claim history · arXiv id

1 claims · 1 propertiesShow all properties

arXiv idarxiv_id1

Claim history for arXiv id
Value	Valid from → to	Status	Source	Confidence	Extractor
2306.13732	11 Sept 2026 21:03 UTC → current	current	arXiv (Atom API + RSS)T1	high	deterministic

Claims are temporal and append-only: a new observation closes the previous claim (valid_to) instead of overwriting it. Conflicting claims from different sources are kept side by side and flagged — never averaged. Methodology →

11 Sept 2026
New paperPaperReinforcement Learning with Temporal-Logic-Based Causal Diagrams
New paper: Reinforcement Learning with Temporal-Logic-Based Causal Diagrams
arxiv
rss.arxiv.org

Full timeline →

Source documents
Source	Document	Type	Tier	Last observed	Snapshots
arXiv (Atom API + RSS)	rss.arxiv.org/rss/cs.AI	feed	T1· Official	2 h ago	1

Tier 1 = official/primary, 2 = quality secondary, 3 = community, 4 = unverified. Every snapshot is archived; see all sources and the methodology.