GN0: A Unified Visual-Language Navigation Framework

Toward a Unified Paradigm for Generation, Evaluation, and Policy Learning in Visual-Language-Navigation

Xinhai Li ¹ Xiaotao Zhang ^1,2 Yuehao Huang ^1,3 Jiankun Dong ¹ Tianhang Wang ^1,4 Sunyao Zhou ^1,5 Yunzi Wu ^1,4 Chengnuo Sun ^1,6 Yunfei Ge ¹ Qizhen Weng ¹ Chi Zhang ¹ Chenjia Bai ¹ Xuelong Li ¹

¹ Institute of Artificial Intelligence (TeleAI), China Telecom ² Shanghai Jiao Tong University ³ Zhejiang University
⁴ Tongji University ⁵ Fudan University ⁶ Jiangsu University

Equal Contributions Project Leader Corresponding Authors

Abstract

Embodied navigation connects intelligent agents with the physical world and serves as a fundamental capability for achieving general robotic intelligence. However, the availability and quality of navigation data have long limited the generalization ability of Vision-and-Language Navigation (VLN) systems and their capacity to handle long-horizon tasks. To bridge this gap, we curate diverse 3D scene resources and develop an automated pipeline for large-scale navigation data generation, resulting in the GN-Matrix dataset. Building upon a 3D Gaussian Splatting (3DGS) rendering engine, we further introduce a high-fidelity simulation platform that supports interactive roaming and collision-aware navigation. Furthermore, we propose GN-Bench, the first benchmark to provide BEV-based evaluation, incorporating dynamic 3DGS avatars to establish new standards for human-robot interaction evaluation.

To fully leverage the interactive nature of the simulator, we develop an RL-driven navigation foundation model, Break and Establish (BAE), after supervised learning, DAgger exposes the model to rollout-induced states and recovery behaviors beyond idealized offline demonstrations, breaking the narrow expert-centric distribution inherited from supervision and providing a better starting point for downstream RL exploration. This unified VLN paradigm seamlessly integrates both map-based and map-free settings, enabling a broad spectrum of tasks including instruction following, human following, and goal navigation. To the best of our knowledge, this work is the first to formalize high-fidelity 3DGS-rendered Bird’s Eye View (BEV) representations as a compact and efficient memory mechanism, thereby unlocking the latent spatial reasoning capabilities of Vision-and-Language Models (VLMs). The dual-system architecture of GN0 ensures both flexibility and high-efficiency deployment in real-world scenarios.

Extensive evaluations on GN-Bench and VLN-CE, encompassing both quantitative and qualitative analyses, demonstrate that the proposed GN0 significantly outperforms existing state-of-the-art VLN methods. Overall, this work, GN-Matrix, presents a unified framework spanning data, simulation, and learning, offering new insights for advancing embodied navigation in both academic research and industrial applications.

GN0

Break and Establish (BAE) is a unified embodied navigation foundation model for instruction following, human following, and goal-directed navigation. It supports FPV, BEV, and hybrid visual inputs, while jointly modeling both discrete action sequences and continuous trajectories within a single framework. At its core, BAE adopts a trajectory tokenization scheme that unifies planning representation and improves training efficiency across diverse navigation settings.

Combined with high-fidelity rendered data and supervised fine-tuning, this design effectively unlocks the navigation capability of the underlying vision-language model. A key component is our closed-loop post-training pipeline: after supervised learning, DAgger exposes the model to rollout-induced states and recovery behaviors beyond idealized offline demonstrations, breaking the narrow expert-centric distribution inherited from supervision and providing a better starting point for downstream exploration.

Building on this initialization, DAPO further refines the policy with execution-oriented rewards, enabling more effective exploration, stronger long-horizon decision-making, and more reliable action generation during rollout. Lightweight Action Experts further bridge high-level planning and executable control for practical deployment.

Navigation Data Generation

Navigation Task Generation Pipeline

To support large-scale embodied navigation in photorealistic 3D Gaussian Splatting (3DGS) environments, we construct GN-Matrix , a large-scale navigation dataset covering diverse indoor scenes, dynamic human-centered environments, and benchmark-compatible data formats for multiple navigation settings. To build GN-Matrix, we develop a unified data generation pipeline that integrates multi-source 3DGS scene assets, animated 3DGS human avatars, task synthesis, and multimodal annotation. This pipeline is designed not only to increase the scale of navigation data, but also to improve scene diversity, task coverage, and realism.

The key characteristics of GN-Matrix are summarized as follows:

High-quality 3DGS scene foundation. GN-Matrix is built on visually faithful 3DGS scene representations, providing realistic observations for embodied navigation in indoor environments.

Diverse scene assets. The dataset combines InteriorGS, handcrafted large-scale scenes, and WorldGrow-expanded scenes to improve scene diversity, layout complexity, and navigational difficulty.

Dynamic human avatars. Beyond static environments, GN-Matrix includes animated 3DGS human avatars with diverse appearances and motions, enabling human-centered embodied navigation settings.

Expanded task coverage. In addition to traditional long-horizon goal-based navigation tasks, GN-Matrix also supports dynamic task types such as human-following.

Hand-Crafted

TeleAI 30F Scene Reconstruction

Waymo Reconstruction

WorldGrow

Task Generation

Scene Tone Transformation

Avatars

Trajectory Generation

Closed-loop Simulation and Benchmark

GN-Bench is a high-fidelity embodied navigation benchmark built on 3D Gaussian Splatting indoor environments, designed to provide a unified platform for both policy evaluation and data generation under realistic closed-loop interaction. Beyond photorealistic egocentric RGB observations, GN-Bench additionally supports metric-consistent Bird's Eye View (BEV) rendering, enabling a unified benchmark for both map-free and map-conditioned navigation paradigms. The simulator integrates rendering, motion execution, collision handling, and geodesic planning into a single interactive loop, allowing agents to be evaluated under physically constrained online execution rather than static offline replay. Importantly, GN-Bench also provides oracle planning and online relabeling capabilities, which make it possible to construct DAgger-style corrective supervision directly from policy rollouts. This allows the benchmark to serve not only as an evaluation substrate, but also as a practical training environment for studying recovery behaviors, covariate-shift mitigation, and closed-loop policy improvement in embodied navigation.

GN-BAE

GN-BAE w/o Bev

InternNav Succeed

InternNav Fail

CMA Succeed

CMA Fail

Navid Succeed

Navid Fail

UniNavid Succeed

UniNavid Fail

GN-Bench Experiments

On GN-Bench, the Stage-1 pretrained BAE already performs slightly better than strong baselines such as fine-tuned UniNaVid and InternNav S2. With the full closed-loop pipeline, DAgger and DAPO bring substantial gains, improving both SR and SPL by more than 10 points over the Stage-1 model. Our ablations also reveal that BEV understanding does not emerge automatically from Stage 1 alone: models with BEV input can initially underperform their FPV-only counterparts, but after DAgger and DAPO, the advantage of BEV becomes increasingly clear. Finally, we find that directly applying GRPO-style optimization after Stage 1 is unstable. Without the entropy increase introduced by DAgger, the initial per-token exploration entropy remains extremely low (0.003 vs. 0.06 nats/token), causing DAPO training to collapse and even degrade performance below the Stage-1 model.

Evaluation on GN-Bench

Depth denotes depth maps, BEV refers to Bird's Eye View projections, and FPV represents first-person RGB images. Bold indicates the best result and underlining the second best. ^† denotes methods fine-tuned via SFT on GN-Matrix.

Method	Observation			Seen					Unseen
Method	Depth	BEV	FPV	TL	NE↓	OS↑	SR↑	SPL↑	TL	NE↓	OS↑	SR↑	SPL↑
CMA	Yes	No	Yes	2.9	8.3	15.7	12.5	11.9	3.0	8.1	19.6	15.5	14.9
NaVid	No	No	Yes	3.4	7.9	20.1	14.6	12.8	3.5	7.7	20.3	14.5	12.8
UniNaVid	No	No	Yes	4.6	7.9	22.2	15.0	12.5	5.2	7.8	20.7	12.8	10.3
InternNav(S2)	No	No	Yes	3.6	7.4	23.1	18.8	17.5	3.7	7.2	26.7	22.1	20.3
NaVid^†	No	No	Yes	2.7	7.4	19.4	18.8	18.8	2.7	7.1	23.8	23.1	23.0
UniNaVid^†	No	No	Yes	3.7	7.2	24.1	22.5	21.9	5.8	7.5	23.1	20.8	20.2
InternNav(S2)^†	No	No	Yes	2.9	7.1	22.5	22.4	22.4	2.9	6.9	24.9	24.0	23.7
GN-BAE FPV-only	No	No	Yes	5.2	4.9	48.9	46.4	44.7	5.0	5.6	43.6	38.9	37.3
GN-BAE BEV+FPV	No	Yes	Yes	5.2	4.3	59.3	58.6	58.6	4.0	5.8	40.2	38.5	38.2

Training Pipeline Ablation

Ablation study on the validation split. A* SFT denotes supervised training on offline demonstrations, DAgger-SFT incorporates corrective trajectories from policy rollouts, and DAPO performs reward-guided closed-loop refinement. Checkmarks indicate the stages included in each configuration.

GN-BAE training stage ablation for FPV-only and BEV plus FPV observations
Method	Training Stage			Seen					Unseen
Method	A* SFT	DAgger-SFT	DAPO	TL	NE↓	OS↑	SR↑	SPL↑	TL	NE↓	OS↑	SR↑	SPL↑
GN-BAE FPV-only	Yes	No	No	4.1	6.5	28.9	26.1	24.8	4.1	6.4	32.9	29.4	28.1
	Yes	Yes	No	5.0	5.2	45.4	41.9	40.2	3.1	6.8	29.9	28.1	27.3
	Yes	No	Yes	3.0	6.7	27.5	26.5	25.8	4.9	5.7	42.4	37.8	36.0
	Yes	Yes	Yes	5.3	4.9	49.5	46.9	45.1	5.0	5.6	43.6	38.9	37.3
GN-BAE BEV+FPV	Yes	No	No	3.8	6.8	27.2	23.6	22.4	4.0	6.6	31.4	27.6	26.4
	Yes	Yes	No	4.2	5.4	48.0	47.6	47.4	3.1	6.8	30.4	27.8	26.9
	Yes	No	Yes	3.0	6.9	26.9	25.6	25.0	3.2	6.4	33.5	32.4	32.3
	Yes	Yes	Yes	5.2	4.3	59.4	58.7	58.6	4.0	5.8	40.2	38.5	38.2

VLN-CE Experiments

We evaluate GN-BAE on the standard R2R-CE benchmark using the Validation Unseen split. As shown in table below, GN-BAE achieves the best overall performance, reaching 3.50 NE, 75.6 OS, 67.7 SR, and 63.4 SPL using only single-view RGB input. Compared with the strongest prior RGB-only baseline, GN-BAE reduces NE by 17.5%(from 4.24 to 3.50m) while improving SR and SPL by 2.6 and 1.1 points, respectively, indicating better goal localization and trajectory efficiency.

Comparison on VLN-CE R2R Val-Unseen

^* denotes methods using the waypoint predictor from Hong et al. (2022). Bold indicates the best result and underlining the second best.

Comparison with state-of-the-art methods on the VLN-CE R2R validation unseen split
Method	Observation				R2R Val-Unseen
Method	Pano.	Odo.	Depth	S.RGB	NE↓	OS↑	SR↑	SPL↑
HPN+DN^*	Yes	Yes	Yes	No	6.31	40.0	36.0	34.0
CMA^*	Yes	Yes	Yes	No	6.20	52.0	41.0	36.0
GridMM^*	Yes	Yes	Yes	No	5.11	61.0	49.0	41.0
ETPNav^*	Yes	Yes	Yes	No	4.71	65.0	57.0	49.0
ScaleVLN^*	Yes	Yes	Yes	No	4.80	—	55.0	51.0
InstructNav	Yes	Yes	Yes	Yes	6.89	—	31.0	24.0
R2R-CMTP	Yes	Yes	Yes	No	7.90	38.0	26.4	22.7
LAW	No	Yes	Yes	Yes	6.83	44.0	35.0	31.0
CM2	No	Yes	Yes	Yes	7.02	41.5	34.3	27.6
WS-MGMap	No	Yes	Yes	Yes	6.28	47.6	38.9	34.3
ETPNav + FF	No	Yes	Yes	Yes	5.95	55.8	44.9	30.4
Seq2Seq	No	No	Yes	Yes	7.77	37.0	25.0	22.0
CMA	No	No	Yes	Yes	7.37	40.0	32.0	30.0
NaVid	No	No	No	Yes	5.47	49.1	37.4	35.9
MapNav	No	No	No	Yes	4.93	53.0	39.7	37.2
NaVILA	No	No	No	Yes	5.22	62.5	54.0	49.0
UniNaVid	No	No	No	Yes	5.58	53.3	47.0	42.7
StreamVLN	No	No	No	Yes	4.98	64.2	56.9	51.9
DualVLN	No	No	No	Yes	4.05	70.7	64.3	58.5
CorrectNav	No	No	No	Yes	4.24	67.5	65.1	62.3
GN-BAE	No	No	No	Yes	3.50	75.6	67.7	63.4