CoMManD

A Sandbox for LLM Copilot Multi-entity Maneuvers Decision-making in Adversarial Scenes

Copilot Multi-entity Maneuvers Decision-making

Abstract

While Large Language Models (LLMs) have emerged as a copilot that assists humans in dispatching large-scale entities, they frequently suffer from spatial hallucinations in complex maps and disconnections between semantic reasoning and physical execution. To this end, this study introduces CoMManD, a sandbox that unifies multi-faction entity action and semantic strategy gameplay. In CoMManD, the LLM copilot is expected to understand the map environment and swarm distribution, integrate diplomatic game dynamics among different factions, and formulate strategies to accomplish adversarial tasks. A hierarchical architecture for the LLM copilot is designed to decouple macro-cognitive reasoning from micro-kinematic execution. Evaluations reveal a cognitive hierarchy in which multi-agent debate mitigates spatial hallucinations, while cross-modal alignment remains the key bottleneck, with models systematically failing to connect semantic diplomacy with swarm execution under adversarial pressure.

Demo Video

Framework Architecture

A hierarchical LLM-copilot strategy generation architecture that decouples macro-strategy from micro-execution. The Decision Intelligence module sits at the apex, supporting both Single Agent Decision (SAD) and Multi-Agent Debate (MAD) paradigms, with a plug-and-play topology-grounded prediction model.

Decision Intelligence

Translates human commands into strategic directives via SAD or MAD paradigms, with structured Chain-of-Thought reasoning.

Prediction Model

Topology-grounded prediction that couples visual intuition with symbolic toolchains for physically valid strategic forecasts.

Dual-Stream Output

Produces both kinetic unit control for physical entity dispatch and semantic diplomacy for inter-faction social engineering.

Battle Maps

CoMManD features a highly editable terrain system with semantic markers. Maps support heterogeneous 2D/3D units, environmental modifiers (water, sand, wood, stone), and strategic structures (Bases and Towers).

Dual-Faction Scenarios

96 × 96 × 10

Tri-Faction Scenarios

196 × 196 × 10

Battle Demo

Experimental Results

We evaluate four foundation models (Gemini-3-flash-preview, GPT-5.2, Grok-4.1-fast, Qwen-3.5-plus) across four architectural variants using the Snapshot methodology on six cognitive dimensions.

MAD Mitigates Hallucinations

Multi-Agent Debate effectively suppresses spatial hallucinations through decentralized persona-based critique, significantly improving Unit Trait Comprehension for GPT and Qwen.

Prediction Boosts Dispatch

The topology-grounded prediction model provides the highest capability ceiling for multi-front combat, enabling anticipatory context that translates spatial foresight into superior unit dispatching.

Cross-Modal Gap Persists

All models excel at deception detection but systematically fail to connect high-level diplomatic rhetoric with micro-tactical execution under adversarial pressure.

Model-Specific Vulnerabilities

GPT suffers from an "over-confidence" trap rejecting cooperation; Grok experiences context overload with complex prompts; Qwen exhibits persistent architectural blindness to structural immunities.