Multi-Agent Reinforcement Learning via Agent-Specific Preference
Published in IEEE Transactions on Automation Science and Engineering (T-ASE), 2026
Multi-agent reinforcement learning typically relies on carefully engineered global rewards, which are difficult to design when heterogeneous agents exhibit distinct behaviors. MAGPIE replaces global evaluation with agent-specific preferences: a dedicated expert evaluates each agent, and the resulting local reward models are combined through a monotonic aggregation mechanism to form a coherent global learning objective.
Theoretical analysis shows that optimizing the decentralized preferences converges to a Nash equilibrium and that optimizing the aggregate reward model is equivalent to training the equilibrium policy. Experiments on benchmark multi-agent tasks and a sequential production-line task demonstrate performance comparable to reward-engineered baselines, offering a practical alternative when precise global rewards are unavailable.