Publications

📖 You can also find my articles on my Google Scholar profile.

^* Equal contribution.

Journal Articles

[1] Preference-based Multi-Objective Reinforcement Learning
Ni Mu^*, Yao Luan^*, Qing-Shan Jia
IEEE Transactions on Automation Science and Engineering (T-ASE), 2025

Overview
A theoretically grounded multi-objective preference learning paradigm: optimizing the multi-objective preference reward model is equivalent to solving the convex Pareto front in policy space.
arXiv
[2] A Transformer-based Thermal Surrogate Model for Cooling Control in Data Centers
Hanchen Zhou, Ni Mu, Qing-Shan Jia
IEEE Robotics and Automation Letters (RA-L), 2025

Overview
A transformer-based thermal surrogate model that enables efficient cooling control in data centers.
Paper

Conference Papers

[1] Learning Interpretable Index Policies via LLM-Driven Zero-Order Optimization
Yao Luan^*, Ni Mu^*, Qing-Shan Jia
IEEE CASE 2026, 2026

Overview
Learns interpretable index policies by driving zero-order optimization with a large language model. Under review (IEEE CASE 2026).
Paper
[2] COLLIE: Guiding Skill Discovery in Semantically Coherent Latent Space
Yao Luan^*, Ni Mu^*, Hanfei Ge, Yiqin Yang, Bo Xu, Qing-Shan Jia
ICML 2026, 2026

Overview
Unsupervised skill discovery often yields task-irrelevant or hazardous behaviors. COLLIE leverages dense unsupervised data to construct a semantically coherent skill latent space, enabling reliable, training-free guidance from sparse online human feedback.
arXiv · Code
[3] GlobeDiff: State Diffusion Process for Partial Observability in Multi-Agent System
Yiqin Yang^*, Xu Yang^*, Yuhua Jiang, Ni Mu, Hao Hu, Runpeng Xie, Ziyou Zhang, Siyuan Li, Yuan-Hua Ni, Qianchuan Zhao, Bo Xu
ICLR 2026, 2026

Overview
A state diffusion process that handles partial observability in multi-agent systems.
Paper
[4] MrCoM: A Meta-Regularized World-Model Generalizing Across Multi-Scenarios
Xuantang Xiong^*, Ni Mu^*, Runpeng Xie, Senhao Yang, Yaqing Wang, Lexiang Wang, Yao Luan, Siyuan Li, Shuang Xu, Yiqin Yang, Bo Xu
AAAI 2026, 2026

Overview
Model-based RL world models generalize poorly across scenarios. MrCoM extracts scenario information via meta-state regularization and aligns model optimization with multi-scenario policy learning via meta-value regularization.
arXiv
[5] Covering the Pareto Frontier with LLM-Coordinated Interpretable Policy Library
Ni Mu^*, Yao Luan^*, Qing-Shan Jia
IFAC 2026 (23rd IFAC World Congress), 2026

Overview
Covers the Pareto frontier by coordinating an interpretable policy library with a large language model.
Paper
[6] MAGPIE: Utilizing Agent-Specific Preferences for Multi-Agent Reinforcement Learning
Ni Mu^*, Yao Luan^*, Qing-Shan Jia
CAC 2025 (2025 China Automation Congress), 2025

Overview
Leverages agent-specific preferences for multi-agent reinforcement learning.
Paper
[7] LLM-Empowered Knowledge Graph Construction for Optimization Formalization in Smart Manufacturing
Ni Mu, Zeyuan Liu, Yuhang Zhu, Qing-Shan Jia
CAC 2025 (2025 China Automation Congress), 2025

Overview
Uses a large language model to construct knowledge graphs that formalize optimization problems in smart manufacturing.
Paper
[8] DAIL: Beyond Task Ambiguity for Language-Conditioned Reinforcement Learning
Runpeng Xie^*, Quanwei Wang^*, Hao Hu, Zherui Zhou, Ni Mu, Xiyun Li, Yiqin Yang, Shuang Xu, Qianchuan Zhao, Bo Xu
NeurIPS 2025, 2025

Overview
Moves beyond task ambiguity in language-conditioned reinforcement learning.
arXiv · Code
[9] STAIR: Addressing Stage Misalignment through Temporal-Aligned Preference Reinforcement Learning
Yao Luan^*, Ni Mu^*, Yiqin Yang, Bo Xu, Qing-Shan Jia
NeurIPS 2025, 2025

Overview
In multi-stage tasks, preference-based RL suffers from stage misalignment: comparing behaviors from different stages is hard for humans and uninformative for policy optimization. STAIR minimizes the temporal distance between segment pairs so comparisons stay within a stage.
arXiv · Code
[10] Addressing Coupling in Restless Multi-Armed Bandits by Finetuning Whittle Index
Yao Luan, Ni Mu, Qing-Shan Jia
IEEE CASE 2025, 2025

Overview
Addresses arm coupling in restless multi-armed bandits by finetuning the Whittle index.
Paper
[11] Safety-Guaranteed Policy Composition Via Generalized Policy Improvement for Autonomous Vehicles
Ni Mu, Yao Luan, Qing-Shan Jia
IEEE CASE 2025 (Outstanding WiRA Student Paper Award), 2025

Overview
A Generalized-Policy-Improvement-based policy composition with safety and performance guarantees that resolves the performance–safety–sample-efficiency "impossible triangle" for autonomous vehicles. Outstanding WiRA Student Paper Award.
Paper
[12] CLARIFY: Contrastive Preference Reinforcement Learning for Untangling Ambiguous Queries
Ni Mu^*, Hao Hu^*, Xiao Hu, Yiqin Yang, Bo Xu, Qing-Shan Jia
ICML 2025, 2025

Overview
In preference-based RL, humans struggle to compare behaviorally similar trajectory segments, which hurts label efficiency. CLARIFY learns a contrastive objective in the trajectory embedding space to select segment pairs that humans can actually compare.
arXiv · Code
[13] S-EPOA: Overcoming the Indistinguishability of Segments with Skill-Driven Preference-Based Reinforcement Learning
Ni Mu^*, Yao Luan^*, Yiqin Yang, Bo Xu, Qing-Shan Jia
IJCAI 2025, 2025

Overview
To address the indistinguishability of behaviorally similar segments in preference-based RL, S-EPOA uses unsupervised skill discovery to pick segments that differ widely in skill, improving human label efficiency.
arXiv · Code
[14] Preference-Based Multi-Objective Reinforcement Learning with Explicit Reward Modeling
Ni Mu, Yao Luan, Qing-Shan Jia
CAC 2024 (2024 China Automation Congress), 2024

Overview
Extends preference-based multi-objective reinforcement learning with explicit reward modeling.
Paper
[15] Large-scale data center cooling control via sample-efficient reinforcement learning
Ni Mu, Xiao Hu, Qing-Shan Jia, Xun Zhu, Xiao He
IEEE CASE 2024, 2024

Overview
Applies sample-efficient reinforcement learning to large-scale data-center cooling control.
Paper
[16] Integrating mechanism and data: Reinforcement learning based on multi-fidelity model for data center cooling control
Ni Mu, Xiao Hu, Qing-Shan Jia
CAC 2023 (2023 China Automation Congress), 2023

Overview
Combines mechanism models with data via a multi-fidelity model for reinforcement-learning-based data-center cooling control.
Paper
[17] E-MAPP: Efficient Multi-Agent Reinforcement Learning with Parallel Program Guidance
Can Chang, Ni Mu, Jiajun Wu, Ling Pan, Huazhe Xu
NeurIPS 2022, 2022

Overview
Improves sample efficiency in multi-agent reinforcement learning by guiding learning with parallel programs.
arXiv · Code

Preprints

[1] SunTzu: A Long-Horizon Decision Making Benchmark for Large Language Models
Pengbo Shen^*, Yaqing Wang^*, Ni Mu^*, Yao Luan, Runpeng Xie, Senhao Yang, Lexiang Wang, Hao Hu, Shuang Xu, Yiqin Yang, Bo Xu
Preprint, 2025

Overview
A benchmark for evaluating LLMs long-horizon decision making in StarCraft II, together with a hierarchical agent baseline that self-corrects via expert knowledge and self-improves via supervised fine-tuning.
arXiv · Code

Ni Mu

Publications

Journal Articles

Conference Papers

Preprints