超越可达性：交互式导航新基准与智能体MultINav

Navigation Beyond Reachability: The MultINav Framework

Traditional navigation systems, from basic PointGoal tasks to advanced visual language navigation (VLN), have long operated under a hidden assumption: the environment is static. The paths available and the state of objects never change, which creates a clear gap between these models and real-world scenarios. For example, a closed door blocks access to a room until it’s opened, and a closed refrigerator hides a target object inside until it’s opened. These actions are rarely independent—opening a door is often a critical prerequisite to accessing a container, making the sequence of interactions essential to completing navigation tasks.

This gap is what the new MultINav framework is designed to address. The framework lets an intelligent agent decide what type of interaction to perform, which object to engage with, when to act, and how to proceed toward the goal after each action. It unifies two key types of spatial changes: altering connectivity through doors (channel interactions) and altering visibility through containers (like cabinets or drawers), while also accounting for the sequential dependencies between these actions.

Three Core Contributions

1. Formalizing the Problem: MultINav formally defines the interactive navigation task, integrating both the connectivity changes from doors and visibility changes from containers, along with their sequential relationships.
2. Creating the Benchmark: A 3,000-episode dataset built on existing environment models and navigation task seeds, split into three task types: channel (door interactions), container (object visibility changes), and mixed (combined sequences). Each episode labels interactions as either required or unnecessary, with tools to automatically generate tasks and evaluate performance.
3. Building the Agent: The MultINav-Agent operates in three stages using a persistent, state-aware interaction graph. A vision-language model (VLM) selects actions based on connectivity and visibility, validates each interaction, and updates the graph as it progresses toward the goal.

MultINav-Bench: The Dataset

The dataset is constructed using counterfactual checks to ensure accuracy: doors must truly block the shortest path when closed, and targets must be completely hidden inside containers before they are opened. Key details include:

• 3,000 episodes across 639 houses, with 1,000 episodes dedicated to each of the three task types (channel, container, mixed)
• 2,654 labeled required interactions and 346 unnecessary interactions (to measure "unnecessary" actions like opening a door that does not help with the goal)
• Additional data for vision-language pre-training, including multi-view expert demonstrations, 6D end-effector poses, and articulated object states

The three task types are defined by the spatial changes they drive:

• Channel tasks: Interact with hinged doors to change spatial connectivity, such as opening a necessary door to reach a target room, or avoiding a distracting door that doesn’t help with the goal
• Container tasks: Interact with cabinets, refrigerators, or drawers to reveal hidden target objects that were previously out of view
• Mixed tasks: Combine both types, such as opening a door to access a room with a refrigerator, then opening the refrigerator to retrieve the target object

MultINav-Agent: The Three-Stage Process

The agent’s core is a persistent, state-aware interaction graph that stores rooms, interactive objects, candidate targets, plus state estimates, observation confidence, and recent activity. The framework operates in three distinct stages:

1. Perception: Link the same entities across different observations, and use visual properties to estimate object states (open, closed, or uncertain). For doors, the connectivity between two rooms depends entirely on whether the door is open.
2. Action Selection: The vision-language model chooses an action based on whether it affects spatial connectivity or object visibility, using the interaction graph to guide decisions.
3. Validation & Update: After performing an action, verify its physical effect, then update the interaction graph to reflect the new state, before proceeding toward the final goal.

Evaluation Metrics

The framework is evaluated using five key metrics to measure performance:

• SR (Success Rate): The percentage of tasks where the agent reaches the goal while meeting both distance and visibility requirements
• SPL (Success-weighted Path Length): An efficiency metric that combines task success with how closely the agent’s path matches the optimal path
• ISR (Interaction Success Rate): The average proportion of required interactions completed, with partial credit for incomplete actions
• IP (Interaction Precision): The percentage of attempts that produce a verified physical effect on the target object category
• Total Cost: A normalized cost that combines path length, number of actions, and execution time, with a fixed normalization constant to compare performance across different tasks

The framework has been tested on a quadruped robot (Go2) and demonstrates strong performance, outperforming baseline methods across all evaluation metrics.