HeteroPROMPT: A Real-time and Privacy-Preserving Heterogeneous Collaborative Perception Framework

Abstract

Collaborative Perception (CP) improves autonomous systems' awareness of their surroundings by sharing sensor data, intermediate features, and detection results. In real-world deployments, however, collaborating vehicles often use heterogeneous sensors, perception models, datasets, and training domains, creating feature-space shifts that degrade downstream fusion and detection. Existing approaches typically retrain fusion and detection components or introduce modality-specific feature interpreters. These methods scale poorly to newly joining agents and often require access to proprietary metadata, raising privacy concerns. We propose HeteroPROMPT, a real-time and privacy-preserving framework for heterogeneous collaborative perception. HeteroPROMPT rapidly aligns each heterogeneous agent's features with an ego-centric unified feature space through modular prompts and lightweight learning-based tuning, while keeping agent encoders and the collaborative fusion and detection stacks frozen. Its visual prompt-based training and inference modulate Bird's Eye View (BEV) features across channels and spatial locations with low computational overhead. For metadata-free deployment, an autoencoder learns a compact unified representation and extracts modality cues from shared features, enabling real-time modality classification and routing to the appropriate HeteroPROMPT modules without exposing proprietary agent information. Experiments on the OPV2V-H and V2XSet datasets show that HeteroPROMPT improves Average Precision over state-of-the-art heterogeneous CP methods while using orders of magnitude fewer trainable parameters. This offers a scalable and practical CP solution. The proposed modality classifier also predicts the joining agent's modality from compact features with greater than 99.99 percent accuracy during deployment. Code will be available at https://github.com/arminmaleki007/HeteroPROMPT.

Authors 2

Armin Maleki, Hayder Radha

Specification

Official page

arxiv.org/abs/2607.26283

Source:arXiv (Atom API + RSS)·T1·observed 2 h ago·high

Arxiv announce type

replace

Source:arXiv (Atom API + RSS)·T1·observed 2 h ago·high

arXiv id

2607.26283

Source:arXiv (Atom API + RSS)·T1·observed 2 h ago·high

Categories

cs.CV, cs.RO

Source:arXiv (Atom API + RSS)·T1·observed 2 h ago·high

PDF

arxiv.org/pdf/2607.26283

Source:arXiv (Atom API + RSS)·T1·observed 2 h ago·high

Primary category

cs.CV

Source:arXiv (Atom API + RSS)·T1·observed 2 h ago·high

Published

11 Sept 2026

Source:arXiv (Atom API + RSS)·T1·observed 2 h ago·high

Each value shows its source, tier and observation time. Conflicting claims are kept side by side and flagged — never averaged. How AI Atlas records facts →

Provenance