DRG-MAPPO: Hierarchical Dynamic Role-Graph Multi-Agent Reinforcement Learning for Cooperative Air Combat

Abstract

Multi-Agent Reinforcement Learning (MARL) has emerged as a pivotal paradigm for complex decision-making in autonomous systems and air combat. While MARL has demonstrated significant potential in air combat, achieving sophisticated tactical coordination remains a non-trivial challenge. This difficulty is largely attributed to two primary limitations: (1) the absence of structured relational modeling hinders agents from capturing complex, time-varying interactions among battlefield entities; and (2) conventional flat architectures often lack the capability to explicitly model tactical roles, leading to ambiguous task allocation in highly dynamic environments. To address these challenges, we propose Hierarchical Dynamic Role-Graph Multi-Agent Proximal Policy Optimization (DRG-MAPPO), a novel MARL framework that integrates graph-based relational modeling with dynamic role assignment. Specifically, DRG-MAPPO constructs a graph-based representation of battlefield interactions and leverages graph attention mechanisms to extract critical relational features among allies, enemies, and threats. Subsequently, a high-level policy employs a dynamic role assignment mechanism to determine tactical responsibilities (e.g., ``leader'' and ``supporter''). Conditioned on these roles and encoded graph-relational features, a low-level policy executes discrete maneuver actions, facilitating the joint optimization of tactical strategy and collaborative execution. Furthermore, a target-priority auxiliary task is designed to foster the emergence of behaviors such as focus-fire. Experimental results demonstrate that DRG-MAPPO achieves a state-of-the-art win rate of 87%, suggesting that our framework effectively balances relational modeling, interpretability, and optimization stability for cooperative air combat.

Authors 7

Junlin Liu, Chengwei Li, Yang Gao, Hui Chang, Xinchen Zhang, Zhijun Zhao, Hao Zhao

Specification

Official page

arxiv.org/abs/2609.11155

Source:Hugging Face Hub (public pages, model cards, papers)·T2·observed 2 h ago·medium

arXiv id

2609.11155

Source:Hugging Face Hub (public pages, model cards, papers)·T2·observed 2 h ago·medium

Hf paper url

https://huggingface.co/papers/2609.11155

Source:Hugging Face Hub (public pages, model cards, papers)·T2·observed 2 h ago·medium

Hf comments

1

Source:Hugging Face Hub (public pages, model cards, papers)·T2·observed 2 h ago·medium

Upvotes

2

Source:Hugging Face Hub (public pages, model cards, papers)·T2·observed 2 h ago·medium

PDF

arxiv.org/pdf/2609.11155

Source:Hugging Face Hub (public pages, model cards, papers)·T2·observed 2 h ago·medium

Published

10 Sept 2026

Source:Hugging Face Hub (public pages, model cards, papers)·T2·observed 2 h ago·medium

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance