Abstract:This paper presents a multimodal world-state method for task planning and anomaly recovery in low-Earth-orbit (LEO) satellite missions, where heterogeneous observations, tightly coupled hard constraints, and costly global replanning make closed-loop autonomy difficult. The method aligns imagery, telemetry, orbital context, and mission semantics at each decision instant to form a unified world-state layer with modality-availability masks. It then propagates hard constraints over a dynamic task-resource-environment-action graph and uses a learned transition model to assess future reachability. Anomalies and their impact paths are localized through prediction-observation residuals, after which recovery is restricted to the affected temporal window. For training, an offline MILP solver subject to the same hard constraints generates teacher trajectories; behavior-cloning pretraining is followed by closed-loop interaction and advantage-weighted updates in the digital twin. On a self-built LEO digital-twin benchmark, the method achieves task success, recovery success, and constraint satisfaction rates of 89.8%, 81.6%, and 96.3%, respectively, outperforming the strongest baseline by 6.7, 12.2, and 3.3 percentage points while maintaining 59 ms online latency. These results support the four proposed mechanisms within the digital-twin setting.

Abstract:Imitation learning for visuomotor policies has shown strong promise in robotic manipulation, yet it typically relies on a large amount of costly human demonstrations. While synthetic data generation in simulation can partially alleviate this dependence, it requires substantial engineering efforts and remains limited by the sim-to-real transfer gap. A lightweight, plug-and-play data-augmentation framework for robotics is still lacking. To address this gap, we propose a data augmentation framework for real-robot demonstrations, which takes 3D point clouds as its representation and rearranges scene elements via lightweight editing. Prior work has shown that motion-level augmentation (e.g., trajectory variation) is effective for improving spatial generalization; in this paper, we instead focus on skill-level augmentation to enhance stability and generalization during manipulation and physical interaction. Given only a single human demonstration per task, we introduce two mechanisms: Failure Recovery, which turns perturbed trajectories into corrective training signals, and Contact-Preserving Perturbations, which deform only the non-contact regions of a manipulated object while keeping the contact geometry intact. Extensive real-robot experiments across diverse manipulation tasks show that the proposed method significantly improves task success rates and generalizes better to out-of-distribution objects and perturbed executions, confirming the effectiveness of skill-level augmentation.

Abstract:Scene text understanding is a core capability of real-world vision-language tasks and has important application value in scenarios such as visual question answering and image captioning. With the rapid development of multimodal large models, artificial intelligence has achieved breakthrough progress in the field of scene text understanding. However, general multimodal large models still commonly suffer from problems including insufficient utilization of OCR information, OCR recognition errors, and semantic deviations in cross-modal understanding. Therefore, this paper proposes an OCR Prompting Retrieval-Augmented (OP-RA) method for general multimodal large models toward scene text understanding. The method first extracts textual information from images using an OCR system, models the features of OCR tokens from both visual and textual modalities, and associates cross-modal visual-linguistic features of OCR. On this basis, combined with task semantic information, it achieves task-aware OCR retrieval augmentation via similarity calculation, selects the most relevant Top-K key OCR texts from candidates, and inputs them into large models in the form of structured prompts, thereby improving the reasoning and understanding capabilities of the models. The proposed method requires no additional training and exhibits favorable plug-and-play properties. Experimental results on two typical scene text understanding tasks, TextCaps and TextVQA, demonstrate that the proposed method achieves consistent performance improvements across various large models including BLIP-2, LION, and InfiniteVL, verifying its effectiveness and good generalization ability.

Abstract:Embodied intelligence requires agents to generate natural, coherent, and physically plausible motions from language instructions. Text-driven human motion generation has therefore become an important task for bridging language understanding and embodied behavior expression. Existing diffusion-based methods have achieved notable progress in motion naturalness and text-motion semantic consistency. However, they usually apply a unified training and inference strategy to samples with different complexity levels, limiting their ability to handle variations in sample complexity. To address this, this paper proposes CARM, a complexity-assessed routing method for human motion generation. CARM first measures the motion direction matching degree between pelvis features and text features, and the motion posture matching degree between limb features and text features. The two matching scores are then used to compute a sample complexity score and generate routing labels. During training, CARM assigns differentiated timestep sampling ranges according to the sample complexity score, enabling timestep sampling routing. During inference, the complexity score is used to select a shallow, intermediate, or deep output path. Samples with lower complexity scores can exit early, while samples whose complexity remains high at the current stage are forwarded to deeper paths for generation. In addition, auxiliary losses are introduced at the shallow and intermediate exits to improve the stability of multi-exit inference. Experiments on the HumanML3D and KIT-ML datasets show that the proposed method improves motion generation quality, demonstrating its effectiveness.

Abstract:As an important application form of embodied intelligence, connected and automated vehicles (CAVs) can compensate for the limitations of onboard perception under low-visibility and shortened sensing-range conditions through cooperative perception with external information. However, external cooperative information is still affected by transmission delay and packet loss during communication, making it difficult to directly and reliably support current decision-making. Existing studies have mainly focused on perception enhancement, denoising, or communication compensation separately, lacking a unified modeling of the relationships among the physical environment, network links, and cooperative behaviors. This deficiency makes it difficult for decision-making to maintain a dynamic balance between cooperation and communication under multidimensional disturbances, and may even introduce safety risks due to misleading information. These problems become more severe under adverse weather conditions such as rain and snow, where low visibility, perception degradation, and communication signal fluctuations jointly intensify system uncertainty. Therefore, this paper proposes a network-aware vehicle–infrastructure collaborative autonomous decision-making system for adverse weather conditions such as rain and snow. The proposed system comprehensively considers factors such as link delay, packet loss, and information freshness to impose validity constraints and perform credibility evaluation on external cooperative messages. It further constructs a unified temporal window around the current decision instant to achieve spatiotemporal alignment of asynchronous multimodal information. On this basis, a three-layer coupled model consisting of a spatial layer, a vehicular layer, and an information layer is established to jointly incorporate environmental constraints, vehicle interaction relationships, and information transmission quality into the decision-making process. In this way, the utilization weights of external cooperative information can be dynamically adjusted, thereby improving decision safety, accuracy, and robustness in scenarios involving emergency-event warnings under heavy rain and snow, where visibility is obstructed and braking distance is constrained by slippery road conditions.

Abstract:Generating body motions that are synchronized with speech and temporally coherent is essential for enhancing the interactivity of digital humans. This task requires a model to capture both long-term motion evolution and fine-grained local motion details. To address the long-range error accumulation of purely autoregressive models and the high sampling cost of purely diffusion-based models for long-sequence generation, this paper proposes a hybrid generation framework that combines patch-level autoregression with local bidirectional diffusion. Specifically, the latent motion sequence is first divided into fixed-length patches, from which patch-level representations are extracted by an aggregation encoder. A patch-level causal Transformer is then employed to model long-range dependencies across patches and produce patch summary vectors. Finally, a local bidirectional diffusion decoder, conditioned on historical context, is used to recover motion details within each patch. Furthermore, classifier-free guidance based on autoregressive conditional summaries and a temperature-controlled ordinary differential equation (ODE) sampling mechanism are introduced to strengthen speech-conditioned generation and improve sampling controllability. Experimental results on the BEAT2 dataset show that the proposed method achieves an FGD of 4.925, a BC of 7.932, and a Diversity score of 13.24. Compared with EMAGE, the proposed method reduces FGD from 5.512 to 4.925 and raises BC from 7.724 to 7.932, demonstrating improved motion quality and speech synchronization.

Abstract:In research on mobile agents driven by large multimodal models, constructing an internal world model constitutes an important foundation for enabling long-horizon reasoning and planning in graphical user interface (GUI) environments. However, the dynamic and nondeterministic nature of open environments often causes agents to produce state-transition hallucinations, namely, mismatches between internal predictions and actual environmental feedback, which can in turn lead to planning failures and cascading errors. Existing self-reflection mechanisms are often constrained by the model's own cognitive blind spots and lack the capacity for autonomous evolution over prolonged interactions. To address these limitations, this paper proposes a dual-layer cognitive architecture for agents based on the decoupling of System-1 and System-2. System-1 serves as the intuitive execution layer and uses a parameterized world model for rapid state rollouts and decision-tree planning; System-2 serves as the supervisory cognitive layer, providing cognitive oversight through real-time verification of state-transition consistency. When the dynamics residual exceeds a threshold, System-2 intervenes and uses a counterfactual causal attribution mechanism to identify the root cause of the failure, guiding the agent to perform state backtracking and dynamic replanning. Furthermore, by constructing an episodic causal memory bank, the architecture creates a closed-loop data flywheel that drives the continual self-calibration of world model parameters during interaction. Experiments on the multi-domain, cross-application GUI task benchmark developed in this work and in a cross-domain transfer setting show that the architecture can reduce error cascades and facilitate the partial internalization of causal knowledge. As the interaction scale increases, the frequency of System-2 interventions decreases while the independent task success rate increases, providing preliminary evidence of continual self-calibration within the scope of the tested GUI tasks.

Abstract:Image compression fundamentally seeks to maximize information preservation under limited bitrate constraints. Existing methods either treat images as collections of pixels for rate-distortion optimization, or leverage generative model priors to synthesize perceptually pleasing reconstructions, yet neither explicitly models the causal dependency structure underlying image regions, leaving rich intra-image causal information unexploited. This paper proposes a world-model-driven image compression framework that models natural images as structured slices of the physical world governed by causal constraints. We define causal anchors — the primitive world elements encoding the scene's causal dependency structure — and provide an information-theoretic justification for prioritizing bitrate allocation toward these anchors, proving that stronger world model predictive capacity yields greater compression gains from anchor-biased rate allocation. The encoder performs hierarchical causal decomposition via chain-of-thought reasoning to precisely localize causal anchors and construct a non-uniform rate allocation mask; the decoder takes causal anchors as the logical seed and leverages the world model's causal inference capability to perform chain-based scene reassembly for high-fidelity reconstruction. Experiments demonstrate that the proposed method substantially outperforms existing approaches across all metrics at ultra-low bitrates.