Ni Mu

Ni Mu

Ph.D. student, Dept. of Automation, Tsinghua University.

Sitemap

A list of all the posts and pages found on the site. For you robots out there, there is an XML version available for digesting as well.

Pages

Posts

portfolio

publications

Integrating mechanism and data: Reinforcement learning based on multi-fidelity model for data center cooling control

Published in CAC 2023 (2023 China Automation Congress), 2023

Combines mechanism models with data via a multi-fidelity model for reinforcement-learning-based data-center cooling control.

Recommended citation: Ni Mu, Xiao Hu, Qing-Shan Jia. (2023). "Integrating mechanism and data: Reinforcement learning based on multi-fidelity model for data center cooling control." 2023 China Automation Congress (CAC).
Download Paper

S-EPOA: Overcoming the Indistinguishability of Segments with Skill-Driven Preference-Based Reinforcement Learning

Published in IJCAI 2025, 2025

To address the indistinguishability of behaviorally similar segments in preference-based RL, S-EPOA uses unsupervised skill discovery to pick segments that differ widely in skill, improving human label efficiency.

Recommended citation: Ni Mu*, Yao Luan*, Yiqin Yang, Bo Xu, Qing-Shan Jia. (2025). "S-EPOA: Overcoming the Indistinguishability of Segments with Skill-Driven Preference-Based Reinforcement Learning." IJCAI 2025.
Download Paper

CLARIFY: Contrastive Preference Reinforcement Learning for Untangling Ambiguous Queries

Published in ICML 2025, 2025

In preference-based RL, humans struggle to compare behaviorally similar trajectory segments, which hurts label efficiency. CLARIFY learns a contrastive objective in the trajectory embedding space to select segment pairs that humans can actually compare.

Recommended citation: Ni Mu*, Hao Hu*, Xiao Hu, Yiqin Yang, Bo Xu, Qing-Shan Jia. (2025). "CLARIFY: Contrastive Preference Reinforcement Learning for Untangling Ambiguous Queries." ICML 2025.
Download Paper

Preference-based Multi-Objective Reinforcement Learning

Published in IEEE Transactions on Automation Science and Engineering (T-ASE), 2025

A theoretically grounded multi-objective preference learning paradigm: optimizing the multi-objective preference reward model is equivalent to solving the convex Pareto front in policy space.

Recommended citation: Ni Mu*, Yao Luan*, Qing-Shan Jia. (2025). "Preference-based Multi-Objective Reinforcement Learning." IEEE Transactions on Automation Science and Engineering.
Download Paper

Safety-Guaranteed Policy Composition Via Generalized Policy Improvement for Autonomous Vehicles

Published in IEEE CASE 2025 (Outstanding WiRA Student Paper Award), 2025

A Generalized-Policy-Improvement-based policy composition with safety and performance guarantees that resolves the performance–safety–sample-efficiency “impossible triangle” for autonomous vehicles. Outstanding WiRA Student Paper Award.

Recommended citation: Ni Mu, Yao Luan, Qing-Shan Jia. (2025). "Safety-Guaranteed Policy Composition Via Generalized Policy Improvement for Autonomous Vehicles." IEEE 21st International Conference on Automation Science and Engineering (CASE 2025).
Download Paper

SunTzu: A Long-Horizon Decision Making Benchmark for Large Language Models

Published in Preprint, 2025

A benchmark for evaluating LLMs long-horizon decision making in StarCraft II, together with a hierarchical agent baseline that self-corrects via expert knowledge and self-improves via supervised fine-tuning.

Recommended citation: Pengbo Shen*, Yaqing Wang*, Ni Mu*, Yao Luan, Runpeng Xie, Senhao Yang, Lexiang Wang, Hao Hu, Shuang Xu, Yiqin Yang, Bo Xu. (2025). "SunTzu: A Long-Horizon Decision Making Benchmark for Large Language Models." Submitted to NeurIPS 2026.
Download Paper

STAIR: Addressing Stage Misalignment through Temporal-Aligned Preference Reinforcement Learning

Published in NeurIPS 2025, 2025

In multi-stage tasks, preference-based RL suffers from stage misalignment: comparing behaviors from different stages is hard for humans and uninformative for policy optimization. STAIR minimizes the temporal distance between segment pairs so comparisons stay within a stage.

Recommended citation: Yao Luan*, Ni Mu*, Yiqin Yang, Bo Xu, Qing-Shan Jia. (2025). "STAIR: Addressing Stage Misalignment through Temporal-Aligned Preference Reinforcement Learning." NeurIPS 2025.
Download Paper

LLM-Empowered Knowledge Graph Construction for Optimization Formalization in Smart Manufacturing

Published in CAC 2025 (2025 China Automation Congress), 2025

Uses a large language model to construct knowledge graphs that formalize optimization problems in smart manufacturing.

Recommended citation: Ni Mu, Zeyuan Liu, Yuhang Zhu, Qing-Shan Jia. (2025). "LLM-Empowered Knowledge Graph Construction for Optimization Formalization in Smart Manufacturing." 2025 China Automation Congress (CAC).
Download Paper

MrCoM: A Meta-Regularized World-Model Generalizing Across Multi-Scenarios

Published in AAAI 2026, 2026

Model-based RL world models generalize poorly across scenarios. MrCoM extracts scenario information via meta-state regularization and aligns model optimization with multi-scenario policy learning via meta-value regularization.

Recommended citation: Xuantang Xiong*, Ni Mu*, Runpeng Xie, Senhao Yang, Yaqing Wang, Lexiang Wang, Yao Luan, Siyuan Li, Shuang Xu, Yiqin Yang, Bo Xu. (2026). "MrCoM: A Meta-Regularized World-Model Generalizing Across Multi-Scenarios." AAAI 2026.
Download Paper

COLLIE: Guiding Skill Discovery in Semantically Coherent Latent Space

Published in ICML 2026, 2026

Unsupervised skill discovery often yields task-irrelevant or hazardous behaviors. COLLIE leverages dense unsupervised data to construct a semantically coherent skill latent space, enabling reliable, training-free guidance from sparse online human feedback.

Recommended citation: Yao Luan*, Ni Mu*, Hanfei Ge, Yiqin Yang, Bo Xu, Qing-Shan Jia. (2026). "COLLIE: Guiding Skill Discovery in Semantically Coherent Latent Space." ICML 2026.
Download Paper

talks

CLARIFY: Contrastive Preference Reinforcement Learning for Untangling Ambiguous Queries Permalink

Published:

Seminar on CLARIFY (ICML 2025). We introduced a contrastive learning framework that learns a structured trajectory embedding space to quantify human judgment clarity, coupled with a rejection-sampling query mechanism that actively filters ambiguous comparisons. The method significantly improves both human annotation reliability and offline preference-based RL performance.

Enhancing Human Feedback Reliability in Preference-Based Reinforcement Learning Permalink

Published:

PhD proposal defense on enhancing the reliability of human feedback in Preference-Based RL. The work identifies three fundamental causes of unreliable feedback: behavioral indistinguishability, judgment ambiguity, and multi-objective conflicts; and proposes a three-part technical framework: skill-driven PbRL (S-EPOA), contrastive learning-based PbRL (CLARIFY), and preference-based multi-objective RL (Pb-MORL).

Covering the Pareto Frontier with LLM-Coordinated Interpretable Policy Library Permalink

Published:

Conference talk at IFAC 2026, which proposes a vision of self-designing industrial autonomy, and a paradigm shift from tuning opaque neural weights to searching over explicit code logic. We provide a proof-of-concept: a dual-agent LLM loop to autonomously build an interpretable policy library. On industrial multi-objective tasks, it matches deep RL in 30 iterations (<90 min) with fully transparent, deployment-ready code.

teaching