Selected work ICRA 2025 • Atlanta, USA 🇺🇸

MARVEL: Multi-Agent Reinforcement Learning for Constrained Field-of-View Multi-Robot Exploration in Large-Scale Environments

Jimmy Chiun, Shizhe Zhang, Yizhuo Wang, Yuhong Cao, Guillaume Sartoretti

Multi-Agent RLGraph AttentionExplorationConstrained Field-of-View
Paper Code Results Video BibTeX
14.2% shorter trajectories than the strongest baseline vs NBVP, 4 agents
100% success rate across every tested configuration 100 unseen environments, teams of 2–8
90 × 90 m indoor environments explored end to end 5,663 training maps at this scale
< 0.2 s per decision, in every scenario tested decentralized, one policy per robot

Abstract

In multi-robot exploration, a team of mobile robot is tasked with efficiently mapping an unknown environments. While most exploration planners assume omnidirectional sensors like LiDAR, this is impractical for small robots such as drones, where lightweight, directional sensors like cameras may be the only option due to payload constraints. These sensors have a constrained field-of-view (FoV), which adds complexity to the exploration problem, requiring not only optimal robot positioning but also sensor orientation during movement. In this work, we propose MARVEL, a neural framework that leverages graph attention networks, together with novel frontiers and orientation features fusion technique, to develop a collaborative, decentralized policy using multi-agent reinforcement learning (MARL) for robots with constrained FoV. To handle the large action space of viewpoints planning, we further introduce a novel information-driven action pruning strategy. MARVEL improves multi-robot coordination and decision-making in challenging large-scale indoor environments, while adapting to various team sizes and sensor configurations (i.e., FoV and sensor range) without additional training. Our extensive evaluation shows that MARVEL's learned policies exhibit effective coordinated behaviors, outperforming state-of-the-art exploration planners across multiple metrics. We experimentally demonstrate MARVEL's generalizability in large-scale environments, of up to 90 m by 90 m, and validate its practical applicability through successful deployment on a team of real drone hardware.

How It Works

A camera sees a slice of the world, so the planner has to choose where a robot goes and where it looks — jointly, and without a central coordinator.

  1. 01

    Position and orientation, planned together

    Exploration planners normally assume an omnidirectional LiDAR. A drone that can only carry a camera has a constrained field of view, which turns every step into a joint choice of waypoint and heading — and multiplies the action space accordingly.

  2. 02

    An informative graph as the observation

    Viewpoints are sampled across free space and connected by collision-free edges. Each node carries its position relative to the agent, its utility (frontiers visible within sensor range and line of sight), whether a robot occupies it, a guidepost bit marking the A* path to the nearest frontier, and the heading that would see the most frontiers.

  3. 03

    Information-driven action pruning

    Around every node, 36 evenly spaced fields of view are sampled and their observable frontiers recorded. Only the three best headings per neighbouring node survive as candidate actions, which keeps the waypoint-heading action space small enough to learn over.

  4. 04

    Attention over frontiers and orientation

    An encoder–decoder of stacked masked self-attention layers fuses node features with the per-node frontier distribution, then decodes a waypoint–heading policy. The reward sums observed frontiers, a heading term scaled by 0.3 for staying aligned with the A* path, a team-shared frontier term, and +10 on completion.

  5. 05

    Trained once, deployed across configurations

    Training uses 5,663 randomly generated 90 m × 90 m maps with 4 agents, a 10 m sensor range and a 120° field of view; soft actor-critic with an attentive critic, 128-step episodes, and a 10−5 learning rate. Team size, field of view and sensor range then change at test time with no retraining.

The MARVEL policy and critic networks: partial map and frontier distributions feed an attention encoder and decoder that emit a waypoint-heading policy.
Sensor-derived information becomes an informative graph and a per-node frontier distribution; the encoder fuses them, and the decoder scores waypoint–heading pairs. The critic sees the ground-truth graph during training only.
A strip of randomly generated indoor floor plans used for training and testing.
A sample of the 5,663 randomly generated 90 m × 90 m maps the policy trained on.

Results

Every planner was run in the same 100 unseen environments. Switch the metric to see where the differences come from; the numbers behind each chart are one click away.

Hardware Validation

The learned policy was flown on a team of Crazyflie 2.1 drones in a 4 m × 4 m arena, with sensor coverage simulated from live positions so the map grew as the real drones moved.

Four drones flying the learned policy, each labelled with the agent it is executing. The runs confirmed that the policy transfers to physical drone dynamics, not just to the simulator it was trained in.

Paper Video

BibTeX

@inproceedings{chiun2025marvel,
  title={MARVEL: Multi-Agent Reinforcement Learning for constrained field-of-View multi-robot Exploration in Large-scale environments},
  author={Chiun, Jimmy and Zhang, Shizhe and Wang, Yizhuo and Cao, Yuhong and Sartoretti, Guillaume},
  booktitle={IEEE International Conference on Robotics and Automation (ICRA)},
  year={2025},
  pages={11392-11398},
  doi={10.1109/ICRA55743.2025.11127700}                              
}

Related Work