Context-Aware Deep Reinforcement Learning for Autonomous Robotic Navigation in Unknown Area
Deep RLGraph AttentionMapless NavigationReal-Time Planning
0.18–0.22 s
to plan one step, at every complexity tier
83–88% less than D* lite
15–32%
faster end to end than D* lite
planning time is where the gap opens
34%
shorter route than FAR across seven goals
1,367 m against 2,060 m in Gazebo
150
randomised dungeon scenarios tested
50 each at simple, medium and complex
Abstract
Mapless navigation refers to a challenging task where a mobile robot must rapidly navigate to a predefined destination using its partial knowledge of the environment, which is updated online along the way, instead of a prior map of the environment. Inspired by the recent developments in deep reinforcement learning (DRL), we propose a learning-based framework for mapless navigation, which employs a context-aware policy network to achieve efficient decision-making (ie, maximize the likelihood of finding the shortest route towards the target destination), especially in complex and large-scale environments. Specifically, our robot learns to form a context of its belief over the entire known area, which it uses to reason about long-term efficiency and sequence show-term movements. Additionally, we propose a graph rarefaction algorithm to enable more efficient decision-making in large-scale applications. We empirically demonstrate that our approach reduces average travel time by up to 61.4% and average planning time by up to 88.2% compared to benchmark planners (D* lite and BIT) on hundreds of test scenarios. We also validate our approach both in high-fidelity Gazebo simulations as well as on hardware, highlighting its promising applicability in the real world without further training/tuning.
How It Works
The robot knows where it is going but not what stands between it and the goal. Every step is a bet about the shape of the space it has not seen yet.
-
01
Mapless, but not blind
No prior map, no pre-built roadmap: the planner samples viewpoints in the space it has observed so far and connects them into a graph it can reason over, extending that graph as the LiDAR reveals more.
-
02
Attention over context, not just distance
A deep reinforcement learning policy with graph attention weighs the environment’s structure against the pull of the destination, which keeps the robot curious about unknown space without wandering — the failure mode that lengthens the sampling-based baselines’ routes.
-
03
Graph rarefaction keeps it real-time
Sparsifying the graph is what makes a step cost roughly 0.2 s. Removing it costs 136% more planning time per step and 8% more distance travelled in the large Gazebo world.
-
04
Neighbourhood size is a dial, not a retrain
The model was trained with each node connected to its 20 nearest neighbours, then run unchanged at k = 10 and k = 5. At k = 10 it still beats FAR on every criterion; at k = 5 the graph gets too sparse and routes degrade by 254 m.
-
05
Onto a real vehicle
A ground vehicle carrying a Leishen C16 LiDAR ran the policy through a 60 m Ă— 15 m laboratory strewn with chairs, boxes and tripods, with a local planner turning each published waypoint into motion commands.
Results
Randomised dungeon maps first, then one large Gazebo world with seven successive goals, then the lab.
A full run, with the graph growing as the robot uncovers the space.
Hardware Validation
A 60 m Ă— 15 m laboratory with randomly placed obstacles, traversed through a series of predefined points on partially observed point-cloud data.
Paper Video
BibTeX
@inproceedings{liang2023context,
title={Context-aware deep reinforcement learning for autonomous robotic navigation in unknown area},
author={Liang, Jingsong and Wang, Zhichen and Cao, Yuhong and Chiun, Jimmy and Zhang, Mengqi and Sartoretti, Guillaume Adrien},
booktitle={Conference on Robot Learning},
pages={1425--1436},
year={2023},
organization={PMLR}
}