SAGEA Socially-Aware Generative Engine for Heterogeneous Multi-Agent Navigation
Controlled cases: bottleneck yielding, multi-robot crossing, vehicle aware clearance and a dense team merge. Dashed red is the unguided robot path, solid green the guided one.
Generate every agent’s future together. Correct only the robot. Tune safety against accuracy without retraining.
What this paper does
Safe and socially compliant navigation in open human-robot environments requires robots to reason about heterogeneous participants with different dynamics, autonomy levels, and social roles. Existing trajectory prediction and planning methods often rely on homogeneous interaction assumptions or enforce only geometric collision constraints, making it difficult to jointly model asymmetric interactions, coupled prediction-planning, and soft social norms. This paper proposes SAGE, a socially-aware generative engine for heterogeneous multi-agent navigation.
SAGE represents robots and surrounding entities as a directed heterogeneous graph and employs a Heterogeneous Graph Transformer (HGT) to encode type-specific asymmetric interactions. Conditioned on the resulting context, a diffusion-based generative module jointly models future entity trajectories and robot trajectory plans. During inference, a training-free safety-social energy guidance mechanism refines sampled robot trajectories using differentiable collision, kinematic, task-progress, and role-conditioned social-compliance terms.
Extensive experiments on real-world (ETH/UCY and SDD) and synthetic datasets verify the effectiveness of SAGE in improving safety and social compliance while maintaining task performance. The proposed guidance mechanism consistently reduces collision and social-violation rates, scales to teams of up to 20 robots, and enables explicit control of the safety-accuracy-task trade-off without retraining. These findings demonstrate the potential of SAGE as a scalable framework for socially-aware multi-agent navigation in complex environments.
One model for the scene, one guide for the robot
Encode who is who
A directed heterogeneous graph and a Heterogeneous Graph Transformer capture type specific, asymmetric interactions between roles.
Generate joint futures
A diffusion module models future entity trajectories and robot plans together, conditioned on the encoded context.
Guide at inference
Differentiable collision, kinematic, task progress and role conditioned social compliance terms refine sampled robot trajectories. No retraining.
Control the trade off
The guidance scale gives explicit control of the safety, accuracy and task trade off without touching the weights.
Encode, generate, guide
A heterogeneous graph encoder conditions a joint diffusion model. Safety social energy guidance then applies a bounded, robot only correction during sampling.
Encode
Robots and surrounding entities become a directed heterogeneous graph. An HGT encoder turns node features and directed relations into context tokens.
Generate
A context conditioned temporal transformer denoises the joint state of entity futures and robot plans.
Guide
A bounded, robot only correction from the safety social energy refines the clean estimate. The sample is re-noised and the loop repeats.
Tested on real and synthetic scenes
robots in the largest team: guidance scales to teams of up to 20
evaluation settings: ETH/UCY, Stanford Drone Dataset and synthetic navigation
retained result CSVs, with protocols and status documented
ETH/UCY uses pedestrian trajectories with a target as robot proxy and speed derived role labels. SDD semantic labels and synthetic settings provide different evidence. Protocols are in the result index.
Three robots (R1 to R3) on a Stanford Drone Dataset crossing, scene hyang_0. Imagery belongs to the Stanford Drone Dataset and follows its terms.
Guidance scale λ against collision rate (CR), PIR, SVR and R-ADE.
Reduction from unguided across robot team sizes.
The CPU demo refines a crossing scene robot trajectory while keeping entity predictions fixed.
Run it yourself
A CPU demo refines a crossing scene robot trajectory with safety social guidance. No dataset or GPU needed.
git clone https://github.com/Larryhul/SAGE.git
cd SAGE
python -m pip install -e ".[dev]"
python -m sage.demo --output outputs/demoBibTeX
@misc{sage_manuscript,
title = {SAGE: A Socially-Aware Generative Engine for Heterogeneous Multi-Agent Navigation},
author = {Hu, Lan and Liwang, Minghui and Zhu, Wenbo and Yi, Xinlei and Hong, Yiguang and Wang, Xianbin and Jiao, Zhenzhen and Hosseinalipour, Seyyedali},
year = {2026},
eprint = {2607.16619},
archivePrefix = {arXiv},
primaryClass = {cs.RO},
url = {https://arxiv.org/abs/2607.16619}
}